A multi-modal physiological assessment device and method enables the simultaneous recording and then subsequent analysis of multiple data streams of biological signal measurements to assess the health and function of the brain. The multi-modal assessment system includes at least one channel of EEG brainwave data in combination with cognitive information that provide a two-dimensional data stream of (x(t), y(t)) of cognitive information; voice recordings; motion, position, and stability data; galvanic skin conductance; temperature of the subject; pulse-oximetry data, cerebral blood perfusion data, vaso-motor reactivity data, and the like. The collected data is processed to construct candidate features extracted from multiple biological sensor data streams and correlated with multi-modal signatures to identify data indicative of brain health, disease and injury.