A bone anchoring device with a shank (2, 102, 202) to be anchored in a bone or vertebra, a head (3, 103, 203), and a receiving part (4) receiving the head (3, 103, 203) for connecting the shank (2, 102, 202) to a rod is provided. The shank (2, 102, 202) and the head (3, 103, 203) are separate parts. The head (3, 103, 203) has a bore (31, 131, 231) with a cylindrical inner surface, a ring-shaped groove (34, 134, 234) provided on the inner surface, and at least one slit (35, 135, 235) extending from the inner surface to the outer surface (30, 130, 230) of the head (3, 103, 203). The shank (2, 102, 202) has a first portion (21, 121, 221) for anchoring in the bone or vertebra and a second portion (23, 123, 223) with a cylindrical outer surface adjoining a free end (27, 127, 227). A projection (7, 107) is provided on the cylindrical outer surface which engages with the ring-shaped groove (34, 134, 234) of the head (3, 103, 203), when the cylindrical second portion (23, 123, 223) of the shank (2, 102, 202) is inserted in the bore (31, 131, 231) of the head.