Abstract
Cardiovascular care and research increasingly depends on imaging-driven assessment of atherosclerosis which is an inflammatory disease marked by heterogeneous plaques that evolve toward clinical events such as myocardial infarction and stroke. Accurate, reproducible phenotyping of key plaque constituents (e.g., calcification, lipid-rich necrotic cores, fibrous caps, and intraplaque hemorrhage, IPH) remains central to prognosis and therapy selection, yet conventional pathology workflows rely on stain-specific, manual review that is time-consuming, variably reproducible, and often siloed by modality. This dissertation develops and evaluates an end-to-end, deep learning–based framework that integrates whole-slide histology, multiple stains, and transcriptomic measurements to (i) improve phenotyping, focusing on IPH, and (ii) infer spatial gene expression from routine histology augmented by bulk RNA-seq. The overarching aim is to reduce subjectivity, raise fidelity, and bridge morphology with molecular state for decision support in cardiovascular medicine.
Using a large carotid endarterectomy (CEA) cohort (∼3700 specimens) stained with hematoxylin and eosin (H&E) and complementary immunohistochemical and special stains (e.g., Picrosirius Red, CD68, EVG), I constructed an end-to-end pipeline that ingests raw whole-slide images (WSIs) and outputs standardized phenotypes and spatially resolved molecular maps. Slides are segmented and tiled over tissue, and tiles are embedded with self-supervised encoders pretrained on histopathology (e.g., ViT/DINO-family or comparable encoders). To capture context beyond single tiles, the framework fuses multi-resolution evidence and leverages attention-based multiple instance learning (MIL): instance-level features (tiles or cluster tokens) form slide-level bags, and attention pooling emphasizes informative regions without manual ROI selection. This produces two complementary outputs: (1) probabilistic phenotypes (e.g., IPH presence/extent) and interpretable attention heatmaps, and (2) per-tile predictions of gene expression used to reconstruct spatial expression fields that mimic spatial transcriptomics.
For IPH phenotyping, the attention-MIL model is trained on expert-reviewed labels and evaluated against traditional histopathological assessment. The attention maps localize hemorrhagic foci and adjacent microenvironments, offering case-level explanations that pathologists can cross-check on the source WSIs. Integrating multiple stains improves discrimination over H&E alone by encoding collagen organization, elastin integrity, and inflammatory burden not fully visible in a single modality. Multi-resolution inputs further stabilize predictions by combining cellular-scale cues (e.g., erythrocyte remnants, macrophage infiltrates) with tissue-scale architecture (e.g., cap thinning, fibrous cap transitions). Differential gene expression (DGE) analyses downstream of model outputs identify transcripts associated with IPH presence and burden; linear modeling with volcano plots emphasizes genes whose expression tracks with model-highlighted tissue, supporting biological plausibility and facilitating biomarker prioritization.
To link morphology and molecular state, I then extend the framework to predict spatial gene expression from WSIs paired with bulk RNA-seq. The key challenge which is recovering spatial variability from a slide-level bulk profile, is addressed by a context-aware transformer that conditions each target tile on its neighborhood and on global slide context. Specifically, target tile embeddings serve as queries; neighbor embeddings form keys/values; and a global slide representation is injected into queries to encode specimen-level context. Positional information derived from saved tile coordinates regularizes spatial continuity. The network outputs per-tile expression estimates that tile into gene-wise spatial maps at a resolution comparable to typical spatial assays. These maps qualitatively recapitulate known arterial biology. For example, macrophage-associated markers (e.g., CD68-related signals) concentrate in shoulder regions and hemorrhagic niches, while smooth-muscle–related markers (e.g., ACTA2-related signals) emphasize fibrous caps and intimal smooth muscle layers, which provides face validity and actionable hypotheses for targeted validation.
Validation proceeds along three axes. First, technical validation compares IPH predictions to expert annotations and examines calibration and localization via attention overlays, showing that the model focuses on hematoma, cholesterol clefts, and macrophage-rich regions typically implicated in IPH pathology. Second, biological validation correlates model-derived phenotypes with gene expression, identifying differentially expressed genes concordant with expected pathways in hemorrhage, inflammation, and matrix remodeling. Third, spatial face validation assesses whether predicted expression fields align with layer-specific arterial organization and with patterns observed in external spatial datasets when available; exploratory analyses in coronary beds suggest the model captures cap-biased smooth-muscle remodeling and media rarefaction consistent with disease progression. Together, these results indicate that integrating multi-stain, multi-resolution histology with attention MIL yields reproducible phenotypes and biologically meaningful spatial transcriptomic surrogates from routine pathology workflows.
Generalizability is addressed through stain and resolution augmentation, harmonization of pre-analytic variability, and training strategies that mix stains and magnifications to encourage invariance. The attention-MIL formulation is particularly robust to missing stains: when certain modalities are unavailable, the model gracefully falls back to H&E and available channels without catastrophic failure. Cross-validation by site and demographic strata demonstrates that performance is stable across clinically relevant subgroups, supporting portability. Importantly, the entire pipeline is standardized end-to-end, from tiling and feature extraction to model inference and visualization, enabling batch deployment and prospective testing.
This dissertation makes two primary contributions. First, it delivers a practical, interpretable, and modular workflow for atherosclerosis phenotyping, centered on IPH, that reduces reliance on exhaustive manual review and improves consistency across large slide cohorts. The combination of attention-based MIL, multi-stain fusion, and multi-resolution context yields clinically interpretable heatmaps and phenotypes suitable for downstream research and potential integration into quality-assurance pathways. Second, it introduces a method to infer spatial gene expression directly from WSIs paired with bulk RNA-seq, producing high-resolution expression maps that can validate known biomarkers, highlight candidate genes associated with plaque instability, and generate spatial hypotheses for follow-up experiments, without requiring dedicated spatial assays on every specimen.
In sum, the proposed framework advances imaging-molecular integration for cardiovascular pathology by unifying histology, machine learning, and transcriptomics in a single, standardized pipeline. By improving phenotyping precision and mapping molecular heterogeneity at slide scale, it lays groundwork for earlier risk stratification, mechanistic insight into plaque progression, and more targeted therapeutic development. The methods are broadly applicable to other vascular beds and, with minor adaptation, to additional disease contexts in which multi-stain histology and bulk omics are abundant but spatial measurements are scarce.
