Abstract
The objective of this study was to conduct exploratory research to determine the viability of a new model for detecting lack of invariance (LOI) for both item responses and response times. LOI occurs when the property of parameter invariance, which states that item parameters are invariant across examinee populations and person parameter are invariant across sets of items, is violated (Rupp & Zumbo, 2006). LOI can be present for item responses, which refers to whether an examinee answered an item correctly or incorrectly, as well as item response times, which refers to how much time an examinee spent answering an item. LOI can be problematic as it has consequences for validity and fairness of the test (Gierl, 2005).
Currently, much of the research on LOI is relegated to studies of item responses (see Demars, 2004a, 2004b; Lord, 1977; Rupp & Zumbo, 2006; Wells, Subkoviak, & Serlin, 2002). Little research has been done on LOI for response times (Demars & Wise, 2010; Klein Entink, 2009; van der Linden, Schnipke, & Scrams, 2007), and no research has looked at LOI for item responses and response times at the same time. As such, this study evaluated a model, multiple indicator multiple cause model for detecting LOI in item responses and response times (MIMIC-IRTRT) that can examine LOI for both item response and response time simultaneously.
This study is conducted in two parts consisting of both a simulation and extant data analysis. In these studies, only uniform LOI was examined. In the simulation study, number of items, correlation between person ability and speed, number of LOI items, type of LOI, and magnitude of LOI were manipulated. In the extant data analysis, high-stakes, college-level, health profession exam data that was suspected to possess compromised test items was analyzed with the MIMIC-IRTRT model. The results from both the simulation and extant data study provide support for the use of the MIMIC-IRTRT model in detecting LOI.
