The present invention relates to a system for determining a vital sign of a subject by which the likelihood of generating and outputting false alarms is reduced. The system comprises a vital sign processor that processes said vital sign information signal measured by a sensor attached to a subject to obtain a vital sign of said subject, an image analysis unit that detects motion of a marker attached to said sensor from image data obtained by an imaging unit from at least an imaging region containing said sensor, and an alarm unit that generates and outputs an alarm signal based on the measured vital sign information signal and/or obtained vital sign and on the detected motion of said marker.