A polyaxial bone anchoring device includes a receiving part configured to be pivotably connected to a head of an anchoring element, the receiving part having a channel for receiving a rod and an accommodation space having an opening for accommodating the head, and a sleeve-like insert piece having a sleeve axis and a spherical segment-shaped outer surface portion. The insert piece is configured to be positioned around a portion of the head in the receiving part and to pivot in the receiving part, and includes a first part and a separable second part configured to be connected with the first part by a connection structure configured to permit translational movement between the first part and the second part in a direction transverse to the sleeve axis, while preventing movement between the first part and the second part in a direction parallel to the sleeve axis.