Enhancing Monocular Obstacle Detection in Autonomous Racing with Auto-Labeled Camera Images and LiDAR Detections

Labeling data for supervised object detection methods for autonomous vehicles is costly and time-consuming. Although many datasets like KITTI and NuScenes exist for autonomous driving, when data for a new domain is introduced, such as racing vehicles in the Indy Autonomous Challenge (IAC), the data needs to be manually annotated for machine learning models to detect obstacles. This thesis proposes a novel approach to automate the annotation process for unlabeled camera images of IAC racecars. We use detections from one modality, the LiDAR, to first obtain free annotations for camera images. These free annotations are generated by leveraging the LiDAR-camera cross calibration, which is calculated using a checkerboard, to transform LiDAR detections into the image's coordinate frame. Then, we perform transfer learning using only the auto-annotated camera images on ImVoxelNet, a 3D monocular object detector, to detect racecars. We use data from the Indy Autonomous Challenge to demonstrate that we can not only auto-label camera images from the proposed process, but also obtain detections in the new domain of racecars from another modality by training a detector using the auto-annotated images. This thesis highlights an improvement in detection of racecars of over 2000 times compared to the baseline performance of ImVoxelNet.