A medical portable terminal device includes an operation panel section including a touch panel for a first operator placed in a non-sterilized area to operate and configured to be capable of controlling operations of a plurality of medical instruments disposed in the non-sterilized area, an image pickup section configured to pick up an image of a region where a second operator placed in a sterilized area performs operation of a gesture, a recognizing section configured to recognize the operation of the gesture from the picked-up image, an operation-list memorizing section configured to memorize an operation list for causing the plurality of medical instruments to perform a plurality of operations associated with a recognition result, and a control section configured to switch between control of the plurality of medical instruments based on operation of the operation panel section and control of the plurality of medical instruments based on the operation of the gesture.
