A Two-Step Approach for Syllabus Development and Evaluation using Machine Learning such as Doc2Vec based on Eduinformatics

This study examines the adoption of rubrics in grading criteria at Japanese universities. Between 2016 and 2020, the percentage of universities explicitly stating grading criteria in syllabi increased to 98.1%. However, the use of rubrics for all subjects at the undergraduate level only rose from 2.9% to 6.1%, indicating challenges in rubric adoption. A two-step approach was proposed to address faculty members' difficulties in creating rubrics: 1) introducing a reference rubric (R2) based on common learning outcomes in diploma policies, and 2) having faculty members create rubrics for their individual learning outcomes. Successful implementation of this approach was reported at a university where rubrics were introduced for all subjects. Furthermore, the study evaluated the comparison between R2 and syllabi rubrics using doc2vec, pre-trained by Wikipedia text data, and cosine similarity to assess the degree of optimization for educational goals of each departments.