Provided are a processor device, an endoscope system, and a method of operating a processor device capable of calculating only an index value capable of being calculated from an image acquired in a specific situation.An image obtained by imaging an observation target including a structure is acquired. A clarity calculation unit (80) calculates a clarity of the structure on the basis of the image. An index value selection unit (82) selects an index value capable of being calculated on the basis of the clarity of the structure calculated by the clarity calculation unit (80) among a plurality of index values. An index value calculation unit (84) calculates the index value selected by the index value selection unit (82), on the basis of the image.