A robot control system includes a controller device (120) mountable on a medical instrument and including a housing with a trackpad configured to interact with a printed circuit board therein to create electronic signals corresponding with movement of the trackpad. The electronic signals include a point signal and a click signal. An adapter is formed on the housing and is configured to detachably connect the controller device to the medical instrument. A display (118) is responsive to the point signal of the controller device to permit a cursor to be moved on the display to indicate a position to be imaged or moved to. A robot system (144) is configured to respond to the click signal to move one of a robot directly or an instrument held by the robot in accordance with the position to be imaged or moved to.
