A polyaxial bone anchoring device is provided including an anchoring element (1) having a shank (2) for anchoring in the bone and a head (3), the head comprising a spherically-shaped outer surface portion (3a) a receiving part (4) configured to be pivotably connected to said head (3), the receiving part having a first end (4a), a second end (4b), a central axis (C) extending through the first end (4a) and the second end (4b), a channel (42) for receiving a rod (100), an accommodation space (44) for accommodating the head (3), the accommodation space (44) having a lower opening (45) at the second end (4b), and a coaxial passage (41) extending from the first end (4a) into the accommodation space (44), a cap member (6, 6) that is arranged at least partially in the accommodation space (44), the cap member (6, 6) having a first portion (61) that is configured to be positioned at least partially around the head (3), wherein the first portion (61) is expandable in the accommodation space (44) to allow insertion of the head (3) and compressible to exert pressure onto the head (3) a sleeve-like insert piece (5) configured to be positioned at least partially around the first portion (61) of the cap member (6, 6), wherein the sleeve-like insert piece (5) comprises a spherically-shaped outer surface portion (51) and is configured to pivot in the receiving part (4).