Online Archive of University of Virginia Scholarship
Examining Higher-Order Thinking Skills at Scale: Evidence on Measurement, Inequality, and School Turnaround101 views
Author
Wang, John, Education - School of Education and Human Development, University of Virginia
Advisors
Bassok, Daphna, ED-EDLF, University of Virginia
Abstract
Higher-order thinking skills (HOTS)—analytical, reasoning, and problem-solving abilities—are increasingly important for lifelong success. K-12 schools play a crucial role in developing these skills through structured learning opportunities, yet scholars have raised concerns about whether marginalized students have equitable access to environments that foster HOTS. Despite their importance, researchers have struggled to measure HOTS at scale, limiting our understanding of how educational systems promote these skills. Without such measures, it is difficult to identify the extent of inequity or assess whether educational practices and interventions help or hurt these skills. This dissertation addresses this gap by developing a novel approach that codes over 3,500 assessment items from Massachusetts statewide English Language Arts exams (2021-2023) using Webb’s Depth of Knowledge framework and links them to approximately 200 million item-level responses to construct a large-scale subscale of HOTS. The first chapter evaluates the extent of psychometric validity of using this novel approach, finding that while distinguishing HOTS from lower-order thinking skills (LOTS) is challenging, it is feasible under certain testing contexts. The second chapter documents substantial racial and socioeconomic inequality in HOTS, though these disparities are slightly smaller than those observed for LOTS. The third chapter provides causal evidence that turnaround reforms, an intervention designed to rapidly reduce educational inequality in low-performing schools, in Massachusetts improve both HOTS and LOTS with little evidence of compromise. Together, these findings establish a foundation for understanding how educational systems shape higher-order thinking while foregrounding their equitable development for marginalized students, offering valuable insights for researchers, practitioners, and policymakers seeking to support skills increasingly critical for future success.
Wang, John. Examining Higher-Order Thinking Skills at Scale: Evidence on Measurement, Inequality, and School Turnaround. University of Virginia, Education - School of Education and Human Development, PHD (Doctor of Philosophy), 2026-04-30, https://doi.org/10.18130/76tg-qz35.
