A system and method for multi-mode imaging device control includes a manipulator having one or more first joints and one or more second joints and a controller. The one or more first joints are configured to pivot a focal point of an imaging device about a pivot. The one or more second joints are configured to position the pivot within a three-dimensional space. The controller is configured to actuate the one or more first joints in response to sensed movement of one or more input devices until a release of the actuation of the one or more first joints is detected, detect the release of the actuation of the one or more first joints, and actuate the one or more second joints in response to sensed movement of the one or more input devices following the detection of the release of the actuation of the one or more first joints.