Described herein are systems and methods for selecting a stressor based on thermal measurements. In one embodiment, a system includes an inward-facing head-mounted thermal camera (CAM) and a computer. CAM takes thermal measurements of a region on a periorbital area (THROI1) of the user. The computer detects extents of stress based on THROI1, receives indications of potential stressors to which the user was exposed while THROI1 were taken, and selects the stressor, from among the potential stressors, based on the indications and the extents. Optionally, during most of the time the user was affected by the stressor, the effect of the stressor, as manifested via changes to THROI1, was higher than the effects of most of the potential stressors. Optionally, thermal measurements of other regions on the face may also be utilized to detect the extents of stress.