A rod holding device for binding a rod to an osseous structure. This rod holding device includes a holding body, an anchor member for anchoring the holding body to the osseous structure, a closure member for engagement with an engagement portion of the holding body, and a sleeve member with an opening for receiving the rod and a non-spherical outer contact surface for engagement with a complementary inner contact surface of the receiving portion and configured to be locked in a receiving portion of the holding body by engagement of the closure member with the engagement portion of the holding body, and to secure the rod within the opening. The rod holding device can thus be adapted to rods of various sizes by choosing a sleeve member with an opening of appropriate diameter.