An endoscope system including an endoscope which acquires a front field-of-view image and a lateral field-of-view image of an object of observation a detecting section which has a function capable of detecting whether a treatment instrument is used or not in the endoscope based on a notification signal which is outputted when the treatment instrument is inserted into the endoscope and an image processing section. The image processing section generates an observation image including the front field-of-view image and the lateral field-of-view image within the same screen, displays a magnified image of one field-of-view image out of the front field-of-view image and the lateral field-of-view image on a display, and performs adjacent part display processing of displaying only a part of the one field-of-view image which is adjacent to the other field-of-view image or image compression processing of displaying a compressed image of the other field-of-view image on the display.