An embodiment in accordance with the present invention includes a technology to continuously measure patient mobility automatically, using sensors that capture color and depth images along with algorithms that process the data and analyze the activities of the patients and providers to assess the highest level of mobility of the patient. An algorithm according to the present invention employs the following five steps: 1) analyze individual images to locate the regions containing every person in the scene (Person Localization), 2) for each person region, assign an identity to distinguish ‘;patient’; vs. ‘;not patient’; (Patient Identification), 3) determine the pose of the patient, with the help of contextual information (Patient Pose Classification and Context Detection), 4) measure the degree of motion of the patient (Motion Analysis), and 5) infer the highest mobility level of the patient using the combination of pose and motion characteristics (Mobility Classification).