A processor device and an endoscope system capable of calculating an optimal index value in accordance with a diagnosis purpose is provided. A processor device includes an image acquisition unit, a diagnosis purpose acquisition unit, an index value storage unit, an index value selection unit, and an index value calculation unit. The image acquisition unit acquires an endoscope image obtained by an endoscope image-capturing an observation object. The diagnosis purpose acquisition unit acquires a diagnosis purpose. The index value storage unit stores correspondence between the diagnosis purpose and a plurality of index values relating to a structure of the observation object. The index value selection unit refers to the index value storage unit and selects the index value that is used for the acquired diagnosis purpose. The index value calculation unit uses the endoscope image and calculates the selected index value.
