Abstract
Deep image segmentation of biomedical images is growing in prevalence as an essential tool for extracting and analyzing clinical data. Like other machine learning techniques, deep segmentation models contain hyperparameters that can have a significant impact on prediction performance. A considerable amount of time and computational resources are required to tune hyperparameters, so researchers often settle on settings from previous experience. In this dissertation, the performance and feasibility of multiple hyperparameter optimization techniques are evaluated to prove the efficacy of these methods. The evaluated techniques include random search, Bayesian optimization, multi-armed bandit, and two novel approaches. The first novel approach, Random Search with Statistical Reduction (RSSR), was designed to deal with optimization trials runs that contain bimodal-distributed response data. RSSR also enables the inclusion of a user's utility function within the non-parametric statistical testing to reduce the settings space. The second novel approach, Gaussian Mixture with Epsilon Greedy, was designed to ensure continued exploration of the hyperparameter space during long optimization runs.
To evaluate these approaches, three varying biomedical image segmentation datasets are utilized to foster a robust comparison. These datasets include: eosinophil detection in Eosinophilic Esophagitis patients, multiple cell type detection in 3D cardiovascular florescent microscopy, and handwritten vital sign detection in surgical flowsheet graphs. The results from the evaluation revealed that the Gaussian process-based Bayesian optimization consistently produced higher validation set accuracy when computation time is limited and was also the most Pareto efficient option. When ample computation time is available, the RSSR approach is able to find high validation accuracy by effectively reducing the search space. Several hyperparameters such as batch normalization, learning rate, and batch size were proven as crucial in reaching minimum validation loss. With these findings, researchers will be more encouraged to perform hyperparameter optimization on real-world image segmentation problems and will know the most effective techniques to execute the process depending on their situation.
