An image processing device includes an inward state detection unit that detects, on the basis of an inward image acquired by photographing of a user side, a state of outside light in the inward image as an inward outside light state, and an outward state detection unit that detects, on the basis of an outward image acquired by photographing of an opposite side of the user side, a state of outside light in the outward image as an outward outside light state. Also included is a control unit that performs brightness adjustment processing related to brightness of the inward image according to a detection result of the inward outside light state and a detection result of the outward outside light state, and a recognition processing unit that performs gaze detection of the user on the basis of the inward image acquired by performance of the brightness adjustment processing.