The invention provides an improved possibility to determine a (floating) isocenter of a radiotherapy system (100). The radiotherapy system (100) comprises a couch (200), a gantry (300) which is configured to be rotatable around a gantry axis (310) and having a radiation source (320), and at least one radiation imaging device (330, 340). The system (100) further comprises a calibration system (400) comprising at least one first optical detection means (410) mounted at the gantry (300), at least one second optical detection means (420) fixed in a surrounding area of the couch (200) and/or the gantry (300), a set of first fiducial markers (412) which are selectively attachable at the couch (200) at defined positions and configured to be detectable by the first optical detection means (410), and a phantom (430) which is selectively attachable at the couch (200) at a defined position, wherein the phantom (430) comprises a set of second fiducial markers (421) configured to be detectable by the second optical detection means (420) and a set of third fiducial markers (331, 341) configured to be detectable by the radiation imaging device (330, 340). The system (100) further comprises a controlling means (500) which is configured to selectively activate the radiation source (320) and rotate the gantry (300), and, for one or more rotational positions of the gantry (300), to determine a point of intersection of a beam axis of the radiation source (320) and the gantry axis (310) by linking detection data of at least the first optical detection means (410), the second optical detection means (420) and/or the radiation imaging device (330, 340).