An ultrasound observation system according to the present invention includes an image processing unit, a display unit, a multiple input reception unit, an input information determination unit, an operation mode setting unit, and a target region setting unit. The image processing unit generates an ultrasound image based on an ultrasound signal obtained by an ultrasound endoscope configured to transmit ultrasound to a subject as an observation target and receive the ultrasound reflected from the subject. The display unit displays the ultrasound image generated by the image processing unit. The multiple input reception unit is configured to simultaneously receive inputs at multiple points on the ultrasound image displayed on the display unit. The input information determination unit determines the number of inputs received by the multiple input reception unit and positions of the inputs. The operation mode setting unit sets an operation mode related to the ultrasound image based on a result of determination by the input information determination unit. The target region setting unit sets a target region to be processed corresponding to the operation mode set by the operation mode setting unit, based on the positions of the inputs determined by the input information determination unit.
