A bone anchoring device includes a shank to be anchored in a bone or vertebra, a head, and a receiving part receiving the head for connecting the shank to a rod. The shank and the head are separate parts. The head has a bore with a cylindrical inner surface, a ring-shaped groove provided on the inner surface, and at least one slit extending from the inner surface to the outer surface of the head. The shank has a first portion for anchoring in the bone or vertebra and a second portion with a cylindrical outer surface adjoining a free end. A projection is provided on the cylindrical outer surface which engages with the ring-shaped groove of the head, when the cylindrical second portion of the shank is inserted in the bore of the head.