An endoscope system includes: a light source device for sequentially emitting illumination light of three colors； an image sensor having pixels to perform photoelectric conversion on light from an object irradiated with the illumination light and output image signals； an illumination controller for causing the light source device to emit the illumination light of two colors during a one-line exposure period of the image sensor； and a calculation unit for calculating each color component signal for each horizontal line, based on a relational expression using an output value of a horizontal line, among output values of the image signals in a latest frame, a frame located one frame before the latest frame, and a frame located two frames before the latest frame, using each illumination period per one-line exposure period of each color in each frame, and using each exposure period of each color with respect to the horizontal line.
