A virtual, augmented, or mixed reality (xR) session is hosted in a physical environment. Physiological measurements of an individual wearing a Head-Mounted Device (HMD) are captured while the individual participates in the xR session. An environmental context of the xR session is determined based on measurements of environmental properties of the physical environment. Physiological states of the individual are determined based on the physiological data. Multiple physiological states may be determined by independent chains that evaluate independent physiological data streams. The environmental context and the physiological states are combined, according to data fusion techniques such as algebraic voting, to determine a physiological classification of the individual. The xR session is modified based on the physiological user classification.
