Designing small-scale tests: A simulation study of parameter recovery with the 1-PL

This simulation study investigated the recovery of item and person parameters of the one-parameter logistic model for short tests administered to small samples. A potential problem with such small scale testing is the mismatch between item and person location parameter distributions. In our study, we manipulated the match of these distributions as well as test length, sample size, and item discrimination. Results showed the degree of mismatch likely to occur in practice has a relatively modest effect on parameter recovery. As expected, accuracy in parameter estimation decreased as sample size and test length decreased. Nevertheless, researchers investigating small scale tests are likely to view parameter recovery as acceptable if a study has at least 100 subjects and 8 items.