A manipulator system includes a master manipulator configured to send an input command, a slave manipulator configured to operate according to the input command, an image capturing unit configured to acquire an image of an object, a display device placed in front of the operator and configured to display the image acquired by the image capturing unit, a detection device configured to detect the direction of an operators face of the operator with respect to the display device, and a control unit configured to determine whether the direction of the operators face is within a predetermined angle with respect to the display device based on the detection result in the detection device, and to shift an operation mode of the slave manipulator between a first control mode and a second control mode in which an operation is limited more than in the first control mode based on a determination result.
