Neuromorphic Vision Computing

Author: ORCID icon
Park, Minseong, Electrical Engineering - School of Engineering and Applied Science, University of Virginia
Lee, Kyusang, EN-Elec & Comp Engr Dept, University of Virginia

Neuromorphic computing, referred to as brain-inspired computing for big-data processing and accelerating artificial intelligence (AI) computation, has received a significant boost from the emergence of memristors and associated computing algorithms over the past decade. Recent advancements in memristive systems have enabled the integration of sensing and computing on a chip, known as in-sensor computing, leveraging the memory and dynamic processing capabilities associated with synaptic long-term and short-term plasticity. Among the senses, vision plays a pivotal role in information processing, enabling remote sensing for navigation, learning, and communication. While current neuromorphic systems utilizing advanced memristors have primarily focused on two-dimensional (2D) vision applications, akin to human visual perception, three-dimensional (3D) vision is also vital for machines to tackle more complex tasks by obtaining additional depth information. In this dissertation, we present a comprehensive approach to neuromorphic vision computing that encompasses both 2D and 3D information processing in conjunction with artificial vision dynamics. We demonstrate one III-V photodiode and one nonvolatile memristor (1P1R) array capable of visual sensing, memory, and computing functions. This enables in-sensor computing protocols such as in-situ visual classification and encoding, referred to as 2D neuromorphic vision computing. We also introduce a bio-inspired 3D sensing technique utilizing nonvolatile memristors, known as the resistive time-of-flight (RToF) principle, enabling unprecedented 3D neuromorphic vision computing. we lastly achieve dynamic bio-inspired vision by integrating conventional high-electron-mobility transistors (HEMTs) with emerging 2D ferroelectric materials that emulate synaptic plasticity, potentially enabling mixed 2D/3D neuromorphic vision. This multidimensional approach to neuromorphic vision computing paves the way for empowering advanced computer vision and augmented reality applications.

PHD (Doctor of Philosophy)
Neuromorphic computing, Remote sensing, Heterogeneous integration, Memristors
Issued Date: