Neural Approaches To Automated Speech Scoring Of Monologue And Dialogue Responses

We present Neural Network (NN) approaches to the automated assessment of non-native spontaneous speech in a monologic task and a simulated dialogic task. Three attention-based Bidirectional Long Short-Term Memory (BLSTM) Recurrent Neural Networks (RNN) are employed to learn three dimensions (i.e., delivery, language use, and content) of scoring rubrics for the spoken responses. The prompts or turn history information are encoded to low-dimensional vectors by either a BLSTM-RNN or an end-to-end memory network (MemN2N) and used as the conditions of the inputs of the NN for rating the subscore of content. The three subscores are fused together to generate a holistic score. The experimental results show that our approaches significantly outperform the conventional approaches to speech scoring and the correlations of automatically predicted scores with the reference human scores are higher than human-human agreement levels for both tasks.