Safety Monitor and Counterfactual Explanations for Learning-Enabled CPS 34 views

Recent years have witnessed a growing trend of AI and machine learning integrated into cyber-physical systems (CPS) such as medical devices and autonomous vehicles. These systems offer the potential to transform various domains by improving efficiency and decision-making capabilities. However, their increasing complexity and reliance on data-driven models pose new challenges, particularly in ensuring system safety and providing interpretable decision-making processes. Addressing these issues is crucial for enhancing system transparency, fostering user trust, and supporting effective human-machine collaboration. This PhD dissertation addresses these challenges through the development of safety monitoring and counterfactual explanations for learning-enabled CPS. This dissertation includes three main contributions. First, we develop a quantitative predictive monitoring and control framework using Signal Temporal Logic with Uncertainty to enhance safety in human-machine interaction. Bayesian RNNs are employed to model uncertainty in human behavior, while quantitative robustness degrees are used to evaluate the satisfaction of safety requirements. A novel loss function optimizes uncertainty estimation, improving predictive accuracy and enabling more effective adaptive control in high-stakes applications. Second, we propose a method for generating counterfactual explanations in reinforcement learning, identifying alternative action sequences that enhance outcomes while adhering to predefined policies. Finally, we integrate temporal logic specifications into counterfactual explanations by incorporating LTL/STL into reward shaping. This ensures that generated counterfactual trajectories remain consistent with safety requirements while improving reliability in decision-making processes. The proposed approaches are validated through case studies in healthcare and autonomous systems, illustrating their practical applicability and potential to enhance both safety and transparency in learning-enabled CPS.

PHD (Doctor of Philosophy)

Counterfactual Explanation; Reinforcement Learning; Temporal Logic; Cyber-Physical Systems; Predictive Monitor and Control; Reward Shaping

English

All rights reserved by the author (no additional license for public reuse)

2025-12-08