Item Parameter Recovery for the Two-Parameter Testlet Model with Different Estimation Methods

Abstract: The testlet model is a popular statistical approach widely used by researchers and practitioners to address local item dependence (LID), a violation of the local independence assumption in item response theory (IRT) which can cause various deleterious psychometric consequences. Same as other psychometric models, the utility of the testlet model relies heavily on accurate estimation of its model parameters. The two-parameter logistic (2PL) testlet model has only been systematically investigated in the psychometric literature regarding its model parameter recovery with one full information estimation methods, namely Markov chain Monte Carlo (MCMC) method, although there are other estimation methods available such as marginal maximum likelihood estimation (MMLE) and limited information estimation methods. In the current study, a comprehensive simulation study was conducted to investigate how MCMC, MMLE, and one limited information estimation method (WLSMV), all implemented in Mplus, recovered the item parameters and the testlet variance parameter of the 2PL testlet model. The manipulated factors were sample size and testlet effect magnitude, and parameter recovery were evaluated with bias, standard error, and root mean square error. We found that there were no statistically significant differences regarding parameter recovery between the three methods. When both sample size and magnitude of testlet variance were small, both WLSMV and MCMC had convergence issues, which did not occur to MCMC regardless of sample size and testlet variance. A real dataset from a high-stakes test was used to demonstrate the estimation of the 2PL testlet model with the three estimation methods. Keywords: IRT, testlet model, estimation, full-information, limited-information.