Disclosed are a heart rate detection system and a wearable device using the same. The heart rate detection system includes a light emitting unit, a light sensor and a processor. The light emitting unit emits a light to a human body. The light sensor senses a reflected light of the light and accordingly generates a first electric signal. The processor calculates the low-frequency noise signal and the high-frequency noise signal of the first electric signal, and removes them. After that, the processor executes a peak detection with respect to the first electric signal to calculate peak values of the first electric signal, and calculate a heart rate based on the time intervals among the peak values.