Systems and methods for multi-modality data processing are provided. Some embodiments are particularly directed to interpreting gesture-based commands in a multi-modality processing system. In one embodiment, a method for interpreting user input in a medical processing system includes receiving a state designator corresponding to a mode of operation of the medical processing system, where the mode of operation includes a value representative of a modality selected from the group consisting of: IVUS, OCT, pressure, and flow. A list of active commands is generated based on the received state designator. A user input sequence is received from one or more user input devices. The medical processing system correlates the user input sequence to a command of the list of active commands, and the command is utilized to control operation of a component of the system. The list of active command may include a subset of commands common to multiple modalities.
