The invention disclosed herein has for its object to provide a manipulator system and a medical system capable of diverse operations and manipulations in limited space.The manipulator system 1 includes an operating unit 2 that is put by an operator into operation, a manipulator 3 that is operated by the operating unit 2, a system control unit 4 that controls the manipulator 3 in association with the operation of the operating unit 2, and a tapping sensor unit 22 for detecting tapping by the operator. The system control unit 4 makes a control switchover in association with information detected by the tapping sensor unit 22.