An endoscope system includes: illuminating windows configured to emit a first illuminating light to the front and a second illuminating light to the side inside the subject an observation window configured to acquire an image of the subject from the front an observation window configured to acquire an image of the subject from the side an image generation portion configured to generate a first image based on an image of the subject from the front and a second image based on an image of the subject from the side a control portion configured to compare brightnesses of the first image and the second image a polarization filter configured to adjust the amount of at least one of the first illuminating light and the second illuminating light and a drive portion configured to drive a polarization filter on the basis of a result of the brightness comparison by the control portion.