Bringing Assessment-to-Instruction (A2i) Technology to Scale: Exploring the Process From Development to Implementation

Bringing effective, research-based literacy interventions into the classroom is challenging, especially given the cultural and linguistic diversity of today's classrooms. We examined the promise of Assessment-to-Instruction (A2i) technology redesigned to be used at scale to support teachers' implementation of the individualized student instruction (ISI) intervention from kindergarten through third grade. In seven randomized controlled trials, A2i and ISI have demonstrated efficacy. However, the research version of A2i was not scalable. To bring A2i to scale in schools serving linguistically diverse students, we carried out the current study across two phases. This study represents both an exploration of what it takes to bring an educational intervention to scale (Phase 1) and a quasi-experiment on the literacy outcomes of learners whose teachers used the technology (Phase 2). We integrated assessments of vocabulary, word decoding, and reading comprehension; revised the A2i algorithms to account for the constellation of skills English learners (ELs) bring to the classroom; updated the user interfaces and added new graphic features; and improved bandwidth and stability of the technology. Findings were mixed, including several nonsignificant results, a marginally significant intent-to-treat effect on word reading in kindergarten and first grade for English monolingual students and ELs, and one significant interaction effect, which suggested ELs and students with less developed reading skills in second and third grade benefited most from the intervention. With some caution, we conclude that A2i demonstrates potential to be used at scale and promise of effectiveness for improving code-focused skills for diverse learners. Educational Impact and Implications Statement In this study, we outline the process of bringing Assessment-to-Instruction (A2i) technology to scale within kindergarten through third grade classrooms serving linguistically diverse learners. We carried out this research within two interactive phases. Within Phase 1, we worked closely with our school partners to guide the revision of A2i technology to use at scale. In Phase 2, we conducted a quasi-experiment on the literacy outcomes of learners whose teachers used A2i. Overall, our newly designed A2i technology shows promise to use at scale with kindergarten and first grade monolingual students and English learners. Limitations, implications, and future directions are discussed.