The Invention relates to a control device (25, 28, 29) for controlling a robot system (11) with at least one robot arm (14, 18) on which a surgical instrument (15, 19) is secured that has an end effector (17, 21), wherein the control device (25, 28, 29) comprises an imaging system (25, 28) that records the control specification of at least one hand (30L, 30R), evaluates it and converts it to corresponding control commands for one or more components of the robot system (11). In order to simplify the control of the robot system (11) and in particular to make it intuitive, it is proposed that a control unit (25) be provided which determines the orientation and/or the position and/or the degree of opening of the end effector (15, 19) of a surgical instrument (15, 19) as first parameter or first parameters, and moreover determines the orientation and/or the position and/or the degree of opening of at least one hand (30L, 30R) as second parameter or second parameters, and, in the event of a deviation of one or more of the first parameters from the corresponding second parameter, suppresses a manual control of the end effector (17, 21), and, in the event of an agreement of one or more of the first parameters with the corresponding second parameter, frees a gesture control, such that the surgical instrument (15, 19) can be actuated by hand.
