An Adaptive Fuzzy Modeling of Bottom-Up And Top-Down Visual Attention Using Eye Tracking

Bottom-up and top-down are the two fundamental mechanisms of visual information processing, through which humans can efficiently direct visual attention to task-relevant information under noisy visual environments, and thus, spare mental workload. Healthcare providers routinely perform a wide array of visual tasks for retrieval, creation, and exchange of patient information in the form of Electronic Health Records (EHRs). In clinical practice, EHR interfaces originally designed to support such tasks often interfere with clinical workflows, which may overload mental resources and increase the risk of human errors. Currently, no agent intervenes between the users and EHRs in order to oversee human performance and to adapt the interface or information accordingly. The work presented in this thesis develops around these premises and envisions an intelligent agent in clinical human-computer interaction that can reliably support an intended task in a naturalistic task setting by inferring a user's internal status in visual-information processing. The first step toward that vision, the aim is to propose and validate an Adaptive Neuro-Fuzzy Inference scheme (ANFIS) that can distinguish between bottom-up and top-down information processing based on eye gazes. For validation, this study tracked four physicians' eye gazes and interactive behaviors over a few hours while they performed routine clinical tasks on a medical information system. Hour-long time series of eye movements, keystrokes, and mouse movements, along with the display screen image frames, were collected. The long-term eye movements were characterized by using the metrics associated with fixation patterns: a significant shift of fixation, sustained gazing, and fixation trajectory over time. Those gaze based metrics were statistically quantified as model inputs, to train our neuro fuzzy inference model. For performance comparison, three different types of fuzzy membership functions (Bell, triangle, and trapezoidal) were selected and and tested. In terms of Root Mean Square (RMS) error, the Bell membership function showed the best performance. For an initial proof-of-concept, the model was implemented by using one physician’s gaze behaviors, of which the average, machine-learned fuzzy output probability indicated that the physician was mostly veering toward bottom-up visual attention. This individualized, task-specific pattern of visual attention has implications for the design of intelligent interfaces in Information and Communications Technology (ICT) for different physicians to analyze real-world gaze behaviors.