An operator telerobotically controls instruments to perform a procedure on an object at a work site while viewing real-time images of the work site on a display screen. An assistant controls movement of another instrument towards or away from the work site to relieve the operator from such task. Control of the instrument is automatically switched from the assistant to the operator upon the working end of the instrument entering the view on the display screen.