A control device includes an image acquisition section that acquires an image of an object captured by an imaging optical system of an endoscope apparatus, a determination section that determines whether or not an observation area is in focus based on a pixel value of each pixel within the image of the object, the observation area being an observation target area within the image of the object, and an aperture control section that controls an aperture of the imaging optical system based on a result of the determination.