An endoscope system includes: an image storage portion that stores previously acquired three-dimensional image information with respect to a subject； a luminal organ extracting portion that extracts a predetermined luminal organ from the three-dimensional image information； an objective optical system provided in a distal end of an insertion portion of an endoscope； a line-of-sight direction detecting portion that detects a line-of-sight direction of the objective optical system； a direction comparing portion that compares a core line direction of the predetermined luminal organ that is extracted by the luminal organ extracting portion and the line-of-sight direction； and a position and posture information recording portion that records information regarding a position and a posture of the distal end of the insertion portion of the endoscope in a case where an angle that the core line direction and the line-of-sight direction form is equal to or less than a predetermined threshold value.
