Predicting and Improving Takeover Performance by A Context-Aware Deep Learning Based Assistive System

With the Level 3 of automation, drivers are no longer required to constantly drive or actively monitor their driving environments and may engage in activities other than driving. However, drivers will still be required to take control of vehicles as soon as automation reaches its limits. As a result of being decoupled from the operating task for a prolonged time, drivers have difficulty regaining the vehicle control in a timely manner. In order to counter the difficulty of takeovers, various factors affecting takeover performance have been evaluated. However, not all factors have been studied comprehensively, and the results of some factors have been contradictory. Additionally, there's a need for development of computational models that reliably predict drivers' takeover performance from their physiological and driving environment data, and utilize the outcome to inform drivers about the upcoming hazards.

This dissertation sought to address these shortcomings by (1) Examining the effect of cognitive load, situation awareness, stress, traffic density, and lead time on drivers' takeover behaviors (takeover time and quality) and psychophysiological responses (i.e. eye movements, electroencephalography, galvanic skin responses, and heart rate variability); (2) Developing neural network models for predicting drivers’ attention and takeover performance by utilizing their physiological data, vehicle's status, and driving environment; (3) Designing an end-to-end context-aware in-vehicle alert system which notifies drivers in a real-time about the loss of situation awareness using multimodal modalities, and (4) Evaluating the system in critical conditions by conducting human-subject experiments.