Efficient Collection, Evaluation and Deployment of Large Scale Deep Learning Models in Low Resource Natural Language Processing Scenarios

Deep learning in natural language processing revolutionized low-resource domains like education and healthcare with approaches like transfer learning and prompting-based methods. Large language models can generalize to new tasks in low-resource environments; however, domain-specific data collection often beats generalized data for a given model size.

My dissertation aims to take the advances in deep learning and natural language processing and apply them in the context of low-resource domains like education and healthcare. Through this work, we have collected data using model-assisted labeling by improving the annotation speed by up to 33% and efficiently devised a method to determine examples near the decision boundary of deep learning-based classifiers.

Evaluation of Deep Learning models is tricky because seemingly small perturbations to the input data (for example changing the articles in a sentence) can cause the classification label to change. We proposed a differential-geometry-based approach to find examples that are most and least susceptible to this change.

We also explored how to train and deploy transformer-based models in educational scenarios efficiently. We proposed a tensorized adapter approach (using tensor decomposition-based methods) that reduced the number of tunable parameters of Large Language Model without a drop in performance.