An arousal level assessment system includes at least one first sensor, at least one second sensor, and a processor. The processor acquires driving mode information and assesses which of a manual driving mode or an automated driving mode is selected based on the driving mode information. In a case where the manual driving mode is selected, the processor acquires driving information via the at least one first sensor, acquires physiological information of the driver via the at least one second sensor, and assesses arousal level based on the driving information and the physiological information. In a case where the automated driving mode is selected, the processor acquires the physiological information via the at least one second sensor and assesses the arousal level based on the physiological information without referring to the driving information.