A system and method for processing motion-related sensor data for health application examines motion-related signal from a motion sensor located on a subject on a frame-by-frame basis to detect physical activities performed by the subject. For each frame, a detected activity is classified into a category of activities and performance parameters of the detected activity are estimated.