A light source device (2) includes excitation light sources (4, 5), wavelength converting members (12), an input unit (6) and a light source control unit (7). The excitation light sources (4, 5) emit a plurality of excitation light including mutually different spectra. The wavelength converting members (12) are disposed in a common application region of the plurality of excitation light and wavelength-convert the plurality of excitation light emitted from the excitation light sources (4, 5) into light having mutually different spectra. The light source control unit (7) is able to switch a combination of the excitation light sources that are lighted among the excitation light sources (4, 5) based on an observation mode including a normal light observation mode and a special light observation mode to highlight a particular observation target input to thee input unit (6). The light source device (2) uses light emitted from the wavelength converting members as illumination light, and emits the illumination light corresponding to the observation modes from a same emitting portion.