LiteQAIRISC: System-Level Emulation of RISC-V Processor with AI and Mixed-Precision Quantization Extensions

Artificial Intelligence (AI) and machine learning (ML) have gained significant interest in the last few years. Furthermore, the exponential growth of the Internet of Things (IoT) and the new paradigm of Artificial Intelligence of Things (AIoT) in recent years have made everything move towards digitization, increasing the demand for high-performance energy-efficient computing for ML inferences at the edge of IoT and motivating a great number of research efforts to develop AI accelerators. While most of the AI accelerators have been developed and evaluated in isolation with little support for full SoC integration, this thesis presents LiteAIRISC to bridge these research gaps where we present an instructional manual for integrating a RISC-V processor with tightly integrated AI accelerators in an FPGA System of Chip (SoC) framework to enable system-level evaluations in real-world platforms. The LiteAIRISC system was successfully built on the Xilinx VC707 board, Xilinx Alveo U280 board, and Xilinx Kintex 705 board. At the same time, with the wide adoption of inferences on tiny ML or Edge AI platforms, it becomes increasingly demanding to develop optimal models that fit the limited hardware budget. Layer-wise mixed-precision quantization (MPQ) has become prevailing for edge inference since it strikes a better balance between accuracy and efficiency than the uniform quantization scheme. This thesis proposes QRISC to support the efficient post-training MPQ on edge chips. We tightly integrate the quantized inference units as part of the processor pipeline through Instruction Set Architecture (ISA) and micro-architecture co-design.