Fundamental Algorithmic Solutions Informed by Cognitive Assistance for First Responders 188 views

Emergency Medical Service (EMS) Providers perform various emergency protocols on their patients under stressful and noisy conditions. We introduce several fundamental algorithmic solutions which will be useful in creating a cognitive assistant that can be used by the EMS providers to solve several challenges they face. Cardiac arrest is one of the most critical EMS protocols. Cardiopulmonary resuscitation (CPR) is arguably the most important step in cardiac arrest-based emergency protocols. High-quality CPR with the appropriate compression rate and depth is essential for the survival of cardiac arrest patients. In reality, the CPR administered in many cases can be of low quality due to the suboptimal quality of feedback the EMS providers receive during training which is caused by human biases and errors. There is a need for automatic estimation of CPR depth and rate which is critical in providing real-time feedback to the EMS providers. We collected a multi-modal dataset of 24 participants performing CPR which can be used to develop, evaluate, and test algorithms that can be used to estimate CPR performance by estimating the CPR compression depth and rate. We showed that the inertial sensors in a smartwatch can be used to reliably estimate CPR compression depth and rate. It is preferable to estimate the CPR quality using cameras as well since some EMS providers might not be wearing a smartwatch all the time. We showed that a single camera can be used to reliably estimate the CPR rate. However, estimating the CPR depth requires improvements in monocular depth perception. We showed that defocus blur can be used to accurately estimate depth in general scenes. Any visual depth perception technique is sensitive to the changes of the camera. We developed a novel technique to estimate camera parameters of a given new camera which can be used to eliminate this camera dependency. Therefore, our depth estimation technique can be used by a range of different cameras. We collected a second dataset which consists of hand images and their depth maps. This dataset was used to show that our defocus blur-based method can be used to reliably estimate the distance to the hands from the camera. We used this technique to estimate CPR depth. The depth estimation errors were higher than the results from the smartwatch and we identified possible causes for these errors so they can be alleviated in future research. We introduce a novel few-shot-learning technique that can be used to detect activities with just a very small amount of data. This can be used to detect various EMS steps in the future. Our noise-robust emotion recognition techniques can be used to detect vocal emotions under environmental noise. These vocal emotion recognition methods can be utilized in the future by a cognitive assistant to detect if an EMS provider is stressed and potentially provide feedback to alleviate the stress.

PHD (Doctor of Philosophy)

computer vision; emergency response; machine learning; AI; activity recognition

English

2024-06-19