An interference signal processing method includes: dividing, into a measuring light beam and a reference light beam, each of a plurality of luminous fluxes which respectively have predetermined wavelength bands different from each other and which are emitted by sweeping the wavelength in the respective predetermined wavelength bands detecting, for each of the luminous fluxes, an interference light signal between the reference light beam and a reflected light beam which is reflected from a measuring object at the time when the plurality of measuring light beams are irradiated onto the measuring object calculating at least phase information of a plurality of detected interference light signals and using, as a reference, the phase of the interference light signal obtained from the luminous flux of a first wavelength band, and of generating a corrected interference light signal by correcting, based on the phase information, the phase of the interference light signals obtained from the luminous fluxes of the plurality of wavelength bands different from the first wavelength band.