Automated Tracking and Analysis of Aerial Surveillance Data

As video sources from unmanned autonomous vehicles, surveillance cameras, and other platforms have become ubiquitous, robust methods for target detection and tracking are in increasing demand. The major challenge of such big data collection is that once the data are captured, a cumbersome, if not impossible, task remains for a human analyst to mine the collected data for valuable information. Consequently, automated tracking methods are required.

Towards this end, we present several tracking algorithms to tackle a variety of video sequences, along with a trackability measure to analyze video sequences for the purpose of tracking. We are specifically focused on persistent surveillance applications used to study target movements. Accordingly, the first major contribution of this work is that it provides two approaches for automated methods to track multiple targets in persistent surveillance video sequences. In the first approach, we develop an automated tracker for registered (stationary camera) video sequences; in the second approach, we present a new tracker for unregistered video sequences based on the morphological filter. The second major contribution is the introduction of a novel trackability measure that allows the user to quantify the difficulty of tracking in a variety of environments via an assortment of imaging sensors.

First, we demonstrate a tracking algorithm for registered video sequences: the Snake Particle Filter (SPF) tracker. The SPF tracker is applied to two data sets. The first data set is composed of 28 targets where the SPF tracker has on average an RMSE error of 7 pixels in the horizontal direction and 3.5 in the vertical. The second data set had 7 targets with the mean RMSE in the horizontal direction being 4.5 pixels and 4 pixels in the vertical direction.

Second, we develop a novel algorithm for unregistered video sequences. The algorithm is named the Morphological Scale-Space Tracker (MS2T). We compare the MS2T to a SIFT based tracker, the Automated SIFT Tracker (ASIFT2), and also investigate the incorporation of SIFT into the MS2T. For all the methods here we utilized 34 targets and 2 measurements, namely the percentage of tracking and normalized root mean squared error (RMSE). The tracking results show that: a) ASIFT2 has an average of 90% frames tracked and 0.45 (half of target width) normalized RMSE; b) MS2T has an average of 96% frames tracked and 0.27 (less than third of target width) normalized RMSE, and; c) incorporating SIFT into MS2T results in an improvement of 98% frames tracked and 0.28 normalized RMSE.

Lastly, we establish a novel quality measure for tracking, which we named the trackability measure. The trackability measure enables, for the first time, quantification of the difficulty of tracking for a given scenario, based on the target appearance, the target motion, and the video quality. Previous measures considered only video quality. Overall, tracker performance parallels the newly introduced trackability measure in terms of the Spearman correlation.

By developing both automated tracking algorithms and a trackability measure we provide a comprehensive approach to tracking. Furthermore, this dissertation introduces a broad set of tracking methodologies by tracking not only registered video sequences, but also unregistered sequences.