The present disclosure provides a non-invasive, inexpensive and unobtrusive system that enables heart rate (HR) monitoring by addressing the traditionally known issues with face video based systems due to respiration, facial expressions, out-of-plane movements, camera parameters and environmental factors. These issues are alleviated by filtering, pulse modelling and HR tracking. Quality measures are defined which incorporate out-of-plane movements to define the quality of each video frame unlike existing approaches which provide a single quality for the entire video. To handle out-of-plane movement, Fourier basis function is employed to reconstruct pulse signals at affected locations. Bayesian decision theory based method performs HR tracking using previous HR and quality estimates for improved HR monitoring.