Described herein are systems and methods for detecting a physiological response based on thermal measurements while accounting for consumption of a confounding substance such as a medication, alcohol, caffeine, or nicotine. In one embodiment, a system includes a computer and an inward-facing head-mounted thermal camera (CAM) that takes thermal measurements of a region of interest (THROI) on a user's face. The computer receives an indication indicative of the user consuming a confounding substance that affects THROI, and detects the physiological response while the consumed confounding substance affects THROI. The detection is based on THROI, the indication and a model. Optionally, the model was trained on: a first set of THROI taken while the confounding substance affected THROI, and a second set of THROI taken while the confounding substance did not affect THROI.
