Image Segmentation in Histopathology with Limited Labeled Data

Detecting and quantifying cellular features via deep neural networks informs understanding of disease progression in digital histopathology. Biopsies from patients are placed on slides, then stained and digitized to create whole-slide images (WSIs). Medical experts provide pixel-level and image-level annotations that outline cell types and disease information on image patches from the WSI. Segmentation neural networks trained on these annotations provide pixel-wise predictions of cell types and tissue disease. However, due to the large amount of tissue imaged for every biopsy, medical experts only label a small portion of the imagery per patient. This results in a large amount of unlabeled data that standard supervised algorithms cannot use. The main question of this research is: How do we optimize segmentation performance in histopathology in limited labeled data settings?
First, model initialization in limited data settings is addressed. Transfer learning techniques have proven more beneficial than random initialization of model weights in many settings. In transfer learning, a model developed for a specific task is reused as the initial model for a second task with limited labeled data. Many models are pre-trained on natural image sets, such as ImageNet, and fine-tuned on medical images. Additionally, some models are pre-trained on one set of medical images and fine-tuned on another set. However, most models are only pre-trained with ImageNet, and there is no standardized medical equivalent of ImageNet. Thus, what type of model initialization is optimal in limited data settings? My published results show the optimality of model weights pre-trained with ImageNet over those pre-trained with histopathology images when the labeled dataset is small. This allows for a broader range of architectures, saving time and preventing expensive gathering of histopathology data and pre-training.
Second, I consider how unlabeled data can be used to optimize segmentation performance. Pseudo-labeling is an existing semi-supervised learning technique that uses unlabeled data to train a model. Existing techniques utilize confidence and uncertainty quantification to select images for pseudo-labeling from a classification standpoint, but the literature does not extend them to the segmentation context. Furthermore, techniques that use pseudo-labels for segmentation either do not specify how the unlabeled data was selected or use a deterministic threshold. Due to the large amount of unlabeled imagery, using all of the WSI in training is not practical. The literature does not address the trustworthiness of the model’s uncertainty quantification. Thus, the second contribution of this research is to adapt and verify the utility of confidence and uncertainty quantification methods from a classification setting to the segmentation setting and to inform image-level selection. My published results show the importance of assessing the correlation between the image-level uncertainty metric and the model performance on a labeled set as a precondition for using the model to select unlabeled images. My approach enables the prioritization of images that maximize performance and provide trust in the model via an intuitive visualization of uncertainty.
Third, I consider how biases inherent in unlabeled and labeled data can be identified that would hinder the generalization of semi-supervised algorithms. Existing methods address differences in labeled and unlabeled sets but do not provide clinically actionable interpretations. I use Gaussian Mixture Models to cluster the unlabeled and labeled sets to identify sampling and labeling biases and demonstrate the effect of these biases in semi-supervised learning algorithms. My method provides clear interpretability about biases that enables the correct clinical solution, reducing cost and minimizing procedures necessary for the patient.