A medical apparatus has a light source section capable of irradiating an excitation light and a reference light alternately, an image pickup section that picks up images of fluorescence by the excitation light, and reflected light of the reference light, a pixel addition processing section that performs pixel addition processing of making a plurality of pixels in a first image as a fluorescence image one pixel, and generates a second image, an adjusting section that makes adjustment so that a brightness ratio of the second image and a reference light image obtained by picking up of the image of the reflected light has a constant value, and thereby generates a third image signal and a fourth image, an interpolation processing section that applies interpolation processing to the third image, and generates a fifth image, and an image processing section that performs image synthesis of the fifth image and the fourth image.