A driver's physical condition detection device for detecting a physical condition of a driver driving a vehicle includes a vehicle detector configured to detect a change in motion of the vehicle during driving, a driver detector configured to detect a change in motion of a head of the driver, and a physical condition determination portion configured to perform a determination process of determining whether or not the physical condition of the driver is deteriorated, based on the change in motion of the head of the driver with respect to the change in motion of the vehicle during driving.