A Skeleton-Based Study of Gait with Applications in Lidar-Based Gait Recognition and Pathological Gait Identification

The study of human locomotion has been bolstered by automated gait analysis in the computer vision community. For years, gait analysis has been mostly limited to academic labs. The emergence of new modalities and the development of computational hardware that are essential for big data analysis has shifted gait analysis toward more practical methodologies. In recent years, gait analysis has emerged as a leading remote identification method for application in areas such as forensic investigation, surveillance, security, and medical fields.

Among the vision-based gait analysis methods, skeleton-based approaches are amenable for reliable feature compaction and fast processing. Model-based gait recognition methods that exploit features from a fitted model, like a skeleton, are recognized for their view and scale-invariant properties. This thesis investigates two problems associated with gait analysis: gait recognition and classification of gait abnormalities.

In the first part of this thesis, we focus on the application of flash lidar imagery to the gait recognition problem. Among available modalities, the emergence of depth cameras, such as Kinect and lidar that provide range (depth) and intensity simultaneously, has alleviated the computationally expensive model fitting that plays a critical role in many gait recognition studies. The current state-of-the-art model-based gait recognition methods take advantage of the high-quality data provided by Kinect and motion capture (Mocap) systems, which are mostly limited to controlled lab environments. Unlike Kinect and Mocap, the lidar camera is suitable for real-world applications; however, the data collected by lidar are noisy and have a lower associated resolution.

In this thesis, we utilize the data collected by a single flash lidar camera for the task of gait recognition. We seek to address the gait identification problem when a considerable number of feature vectors contain faulty and missing values. In particular, we will present methods to avoid the common practice of data elimination under the described conditions while still achieving high accuracy and precision in gait recognition. We describe filtering mechanisms to correct and interpolate the faulty and missing joint locations in the skeletons. In addition, methods are presented to incorporate the dynamic of the motion in the presence of noisy data. We discuss outlier removal as an alternative method for applications in which data elimination is not an issue and present a modification of Tukey's method for the vector-based attributes. Experimental evaluation demonstrates that joint correction can effectively improve the classification scores in the proposed method and several relevant state-of-the-art approaches.

The second part of this thesis presents skeleton-based methods for the gait anomaly recognition problem. The main contributions in this part involve designing skeleton-based features and presenting end-to-end deep learning models that take minimally processed skeleton joints as the input. Unlike the common two-class or one-class approaches of skeleton-based methods, the proposed model considers a multi-class framework. Therefore, the approach can be easily adapted for a more convenient gait assessment outside clinical facilities. The proposed models are evaluated on three publicly available multi-class skeleton datasets with normal/pathological gait data, and achieve high classification scores in detecting minor gait abnormalities. The results indicate the potential of markerless modalities such as Kinect for designing less costly and more convenient health infrastructures for assisted living. Besides, an automatic and non-invasive gait assessment can further augment the clinical diagnosis for an extensive list of ailments that cause different types of gait disorders.