Online Archive of University of Virginia Scholarship
Improving Explainable AI (XAI) Integration in High-Stakes Environments: Examining the Effect of Workload, Task Priority, and Training on XAI Use486 views
Author
Alami, Abdul Jawad, Systems Engineering - School of Engineering and Applied Science, University of Virginia0000-0002-1651-3500
Advisors
Riggs, Sara, EN-SIE, University of Virginia
Abstract
The rapid evolution of Artificial Intelligence (AI) technologies presents both opportunities and challenges for decision-making in high-stakes environments. This dissertation explores the potential of Explainable AI (XAI) based decision support systems in high-stakes applications such as healthcare and military operations. Through an observational case study and two controlled experiments, we investigate the impact of environmental factors on XAI use, the role of training in promoting effective human-AI teaming, and the challenges associated with integrating XAI systems into professional workflows.
Our research reveals that high workload and low task priority significantly impair task performance and the proper use of XAI systems. We demonstrate that training plays a crucial role in promoting appropriate use of XAI and reducing blind reliance on AI recommendations. Notably, we find that real-time XAI-based training can be as effective as traditional guided training in improving performance and calibrating trust in AI systems.
The dissertation provides critical insights into XAI system integration challenges and opportunities in high-stakes environments. We propose design considerations for XAI systems, including adaptivity to workload and task priority. Our findings contribute to the theoretical understanding of trust calibration in XAI systems and highlight the potential of XAI-based training methods. As AI systems become increasingly prevalent in critical decision-making processes, this research contributes to ensuring effective human-AI collaboration for enhanced safety, efficiency, and ethical considerations.
Degree
PHD (Doctor of Philosophy)
Keywords
Explainable AI (XAI); Human–AI collaboration; High-stakes environments; Automation bias; User-centered design; Human factors; Human–computer interaction
Language
English
Rights
All rights reserved (no additional license for public reuse)
Alami, Abdul Jawad. Improving Explainable AI (XAI) Integration in High-Stakes Environments: Examining the Effect of Workload, Task Priority, and Training on XAI Use. University of Virginia, Systems Engineering - School of Engineering and Applied Science, PHD (Doctor of Philosophy), 2024-08-01, https://doi.org/10.18130/exrr-nr24.
