Development and Pilot of an Instrument to Measure Middle School English Language Arts Teachers' Content and Pedagogical Knowledge of Reading

This dissertation study describes the development and pilot study of an instrument to measure middle school English Language Arts (ELA) teachers’ knowledge of reading, the Middle School ELA Teacher Reading Knowledge (METoR) Survey. While there are measures of teacher knowledge that have been validated for use with elementary school teachers (e.g., Binks-Cantrell et al., 2011; Davis et al., 2021; Hall et al., 2023), no such measure exists for middle school teachers. The METoR survey seeks to measure teacher knowledge in four domains: (1) word knowledge: content knowledge; (2) word knowledge: pedagogical knowledge; (3) reading comprehension: content knowledge; and (4) reading comprehension: pedagogical knowledge.

Items were created using texts found through a systematic review as a reference for evidence-based practices. Items were then sent out to expert reviewers for feedback and revision. Using the revised items, cognitive interviews were conducted with three teachers to ensure clarity and relevance. The final measure consisted of 49 multiple choice questions across the four domains.

Forty-two teachers completed the METoR survey. The overall instrument measured with acceptable reliability, but the individual domains do not. A lack of correlation was found across items within domains. Confirmatory factor structure revealed that no factor model had acceptable fit. Overall knowledge was not significantly related to education level, teaching experience, certification type, or certification in elementary education. The lack of reliability for the items within the subdomains suggested the need to make revisions.