An activity monitor such as a wrist-worn device has an accelerometer which continuously detects motion of the user. The activity monitor also has an on-demand heart rate monitor which is activated by the user touching it from time to time. A calorie expenditure based on the motion of the user can be modified based on a heart rate measurement. Further, a determination can be made as to whether the user has made repetitive motions for a period of time. If the repetitive motions are detected, a calorie expenditure based on the heart rate is determined and compared to the calorie expenditure based on the user motion, and the higher value prevails. A situation is avoided in which the activity monitor underestimates the calories expended, such as when the user is exercising strenuously but the accelerometer indicates relatively little motion, e.g., during strength training.