Online Archive of University of Virginia Scholarship
Efficient Collection, Evaluation and Deployment of Large Scale Deep Learning Models in Low Resource Natural Language Processing Scenarios242 views
Author
Datta, Debajyoti, Systems Engineering - School of Engineering and Applied Science, University of Virginia0000-0003-0581-6116
Advisors
Brown, Don, DS-Research, University of Virginia
Abstract
Deep learning in natural language processing revolutionized low-resource domains like education and healthcare with approaches like transfer learning and prompting-based methods. Large language models can generalize to new tasks in low-resource environments; however, domain-specific data collection often beats generalized data for a given model size.
My dissertation aims to take the advances in deep learning and natural language processing and apply them in the context of low-resource domains like education and healthcare. Through this work, we have collected data using model-assisted labeling by improving the annotation speed by up to 33% and efficiently devised a method to determine examples near the decision boundary of deep learning-based classifiers.
Evaluation of Deep Learning models is tricky because seemingly small perturbations to the input data (for example changing the articles in a sentence) can cause the classification label to change. We proposed a differential-geometry-based approach to find examples that are most and least susceptible to this change.
We also explored how to train and deploy transformer-based models in educational scenarios efficiently. We proposed a tensorized adapter approach (using tensor decomposition-based methods) that reduced the number of tunable parameters of Large Language Model without a drop in performance.
Degree
PHD (Doctor of Philosophy)
Keywords
Natural Language Processing; Differential geometry; Tensor Decomposition; Educational Technology
Datta, Debajyoti. Efficient Collection, Evaluation and Deployment of Large Scale Deep Learning Models in Low Resource Natural Language Processing Scenarios. University of Virginia, Systems Engineering - School of Engineering and Applied Science, PHD (Doctor of Philosophy), 2023-04-26, https://doi.org/10.18130/nxen-x383.