Assessment with computer agents that engage in conversational dialogues and trialogues with learners.

This article describes conversation-based assessments with computer agents that interact with humans through chat, talking heads, or embodied animated avatars. Some of these agents perform actions, interact with multimedia, hold conversations with humans in natural language, and adaptively respond to a person’s actions, verbal contributions, and emotions. Data are logged throughout the interactions in order to assess the individual’s mastery of subject matters, skills, and proficiencies on both cognitive and noncognitive characteristics. There are different agent-based designs that focus on learning and assessment. Dialogues occur between one agent and one human, as in the case of intelligent tutoring systems. Three-party conversations, called trialogues, involve two agents interacting with a human. The two agents can take on different roles (such as tutors and peers), model actions and social interactions, stage arguments, solicit help from the human, and collaboratively solve problems. Examples of assessment with these agent-based environments are presented in the context of intelligent tutoring, educational games, and interventions to help struggling adult readers. Most of these involve assessment at varying grain sizes to guide the intelligent interaction, but conversation-based assessment with agents is also currently being used in high stakes assessments.