The invention relates to a system for monitoring a baby (7), comprising a motion data providing unit (10) for providing motion data (12, 14) indicative of motions within a stream of video images (200), a region of interest providing unit (20) for providing a region of interest (22) in the video images (200), a classifying unit (30) for classifying the provided motion data (12, 14) into at least motion data (12, 14) from inside the region of interest (22, 28) and from outside the region of interest (22, 28), an event detection unit (40) for detecting an event by evaluating motion data (12, 14) from inside the region of interest (22, 28) and motion data (12, 14) from outside the region of interest (22, 28). The system and corresponding method and corresponding computer program allow for a more reliable determination of baby in or out of bed status.