The invention provides a patient monitoring system for monitoring a patient in a bed. A video camera is used for capturing video images of the patient. Video analysis is used to determine and track the position of body parts of the patient including the hands. This analysis is enhanced by using sensors which detect interaction by the patient with pieces of equipment in the vicinity of the bed.