For remote heart rate estimation, a method detects an object of interest (OOI) in each image of a video data and tracks the OOI in each image of the video data. The method identifies a region of interest (ROI) within the OOI and generates a plurality of super pixels from a plurality of pixels in each ROI. The method further generates a super-pixel time series from the plurality of super pixels in each image and removes interfering signals from the super-pixel time series. The method further models the super-pixel time series as a super-pixel model and calculates a heart beat signal from the super-pixel model. The method calculates heart characteristics from the heart beat signal. The heart characteristics include one or more of a heart rate, an inter-beat interval, and a heart rate variability.
