An endoscope system includes a first endoscope having a first imaging part, a second endoscope having a second imaging part, an arm attached to the first endoscope and operable to move the first endoscope, a first position and orientation detector configured to detect a position and orientation of the first endoscope in a predetermined coordinate system, a second position and orientation detector configured to detect a position and orientation of the second endoscope in the predetermined coordinate system, and a control device configured to control an operation of the arm, wherein the control device has, as a control procedure for operating the arm, a first calculating step of acquiring the position and orientation of the second endoscope in the predetermined coordinate system from the second position and orientation detector and calculating a position and direction of an optical axis of the second imaging part, an operation amount-calculating step of acquiring the position and orientation of the first endoscope in the predetermined coordinate system from the first position and orientation detector and calculating an operation amount of the arm for operating the arm so that the first imaging part moves in a direction in which an optical axis of the first imaging part is made to coincide with the optical axis of the second imaging part, and an instructing step of outputting an operation instruction for operating the arm on the basis of the operation amount, and wherein the control device controls the operation of the arm according to the control procedure.
