The technology disclosed herein is directed to an endoscope system. The endoscope system includes an endoscope, a sensor； and a controller. The controller is configured to control the endoscope. The controller includes one or more processors. The one or more processors is configured (i) to extract first feature points from a first image captured by the endoscope, (ii) to extract second feature points from a second image captured by the endoscope after acquisition of the first image, (iii) to estimate a first result of position and orientation of the endoscope based on the first feature points and the second feature points, (iv) to calculate a first estimation accuracy based on the first result. When the first estimation accuracy is greater than a predetermined value, the one or more processors utilizes the first result as position and orientation of the endoscope.
