Efficient Collection, Evaluation and Deployment of Large Scale Deep Learning Models in Low Resource Natural Language Processing Scenarios 261 views

Deep learning in natural language processing revolutionized low-resource domains like education and healthcare with approaches like transfer learning and prompting-based methods. Large language models can generalize to new tasks in low-resource environments; however, domain-specific data collection often beats generalized data for a given model size. My dissertation aims to take the advances in deep learning and natural language processing and apply them in the context of low-resource domains like education and healthcare. Through this work, we have collected data using model-assisted labeling by improving the annotation speed by up to 33% and efficiently devised a method to determine examples near the decision boundary of deep learning-based classifiers. Evaluation of Deep Learning models is tricky because seemingly small perturbations to the input data (for example changing the articles in a sentence) can cause the classification label to change. We proposed a differential-geometry-based approach to find examples that are most and least susceptible to this change. We also explored how to train and deploy transformer-based models in educational scenarios efficiently. We proposed a tensorized adapter approach (using tensor decomposition-based methods) that reduced the number of tunable parameters of Large Language Model without a drop in performance.

PHD (Doctor of Philosophy)

Natural Language Processing; Differential geometry; Tensor Decomposition; Educational Technology

English

2023-04-26