Physiological signal processing systems include a photoplethysmograph (PPG) sensor that is configured to generate a physiological waveform, and an inertial sensor that is configured to generate a motion signal. A physiological metric extractor is configured to extract a physiological metric from the physiological waveform that is generated by the PPG sensor. The physiological metric extractor includes an averager that has an impulse response that is responsive to the strength of the motion signal. Related methods are also described.