An infrared imaging device includes an imaging element including a plurality of infrared detection pixels which are two-dimensionally arranged, a diaphragm, a temperature detection unit that detects the temperature of the diaphragm, and a digital signal processing unit that subtracts, from at least a portion of each of a plurality of captured image data items obtained by capturing images using the imaging element in a state in which an F-number of the diaphragm is set to a plurality of values, a signal value corresponding to an amount of infrared rays which are radiated from the diaphragm and are based on the F-number when each of the plurality of captured image data items is acquired and the temperature detected by the temperature detection unit and combines the plurality of captured image data items after the subtraction to generate composite image data.