A three-dimensional endoscope system includes: an imaging unit configured to capture a first image and a second image having parallax with respect to each other； and an image generation unit configured to generate a display image for stereoscopic viewing by performing image processing on the first image and the second image captured by the imaging unit, wherein the image generation unit generates the display image based on the first image and a third image obtained by replacing a part of the second image with a part of the first image at a corresponding position.
