Provided are an endoscope system, a processor device, and a method of operating an endoscope system capable of suppressing a processing load at a timing when users, such as a doctor, do not desire, and automatically executing processing, such as discrimination, at an appropriate timing when the users, such as a doctor, desire, to acquire information that contributes to diagnosis.An endoscope system (10) comprises an image acquisition unit (54), a discrimination processing unit (82), and a discrimination processing control unit (83). The image acquisition unit (54) acquires an endoscope image obtained by imaging an observation target using an endoscope (12). The discrimination processing unit (82) discriminates a portion having a specific feature in the observation target by performing discrimination processing, using an endoscope image. The discrimination processing control unit (83) controls a start or an end of the discrimination processing on the basis of a change in operation of an endoscope (12) or the endoscope image.
