A sound source separating device includes: a sound collecting unit configured to collect sound signals using a plurality of microphones; a sound source localizing unit configured to localize a sound source on the basis of the sound signals collected by the sound collecting unit; a sound source separating unit configured to separate the sound signals collected by the sound collecting unit into sound sources corresponding to the number of fixed sound sources and fixed sound source directions without using the sound source localization result from the sound source localizing unit; and a speech extracting unit configured to extract speech signals of the fixed sound sources on the basis of the sound source localization result from the sound source localizing unit and the sound source separation result from the sound source separating unit.
