Systems and methods to improve performance, reliability and learning to enhance autonomy of vehicles. Sensors capture human eye movements, hearing, hand grip and contact area on steering wheel, the positions of accelerator and brake pedals from the wall behind them as well as from the foot. Outside event signatures corresponding to human reactions and actions are then extracted form these sensors and correlated to events, status and situations acquired using vehicle and outside environment sensors. These outside event signatures are then used to train vehicles to improve their autonomous capabilities.