A closer look at linguistic complexity: Pinpointing individual linguistic features of science multiple-choice items associated with English language learner performance

Linguistic complexity in test items may jeopardize the validity of interpreting English language learners' (ELLs') test scores as measures of their content knowledge in mathematics and science. Although previous studies have demonstrated the effect of linguistic complexity on ELL performance, few examined individual linguistic features. Particularly, little is known about features operating at the level of the item as a whole, rather than the word or sentence level. This study identified 13 linguistic features likely to impact ELL performance on 162 Grade 5 multiple-choice science items from a large-scale science assessment. After analyzing the frequency and distribution of features, we correlated their presence with items' Differential Item Functioning (DIF) values disfavoring ELLs. Three features were positively correlated at a significant level with DIF disfavoring ELLs, and two features were negatively correlated at a significant level with DIF disfavoring ELLs. The results indicate that certain individual linguistic features may differentially impact ELL performance.