Prerequisite Relation Learning for Concepts in MOOCs

ACL, pp. 1447-1456, 2017.

Cited by: 46|Bibtex|Views66|Links
EI
Keywords:
sentence reference distancewikipedia reference weightcomplexity level distanceprerequisite knowledgevideo reference weightMore(21+)
Weibo:
What prerequisite knowledge should students achieve a level of mastery before moving forward to learn subsequent coursewares? We study the extent to which the prerequisite relation between knowledge concepts in Massive Open Online Courses can be inferred automatically

Abstract:

What prerequisite knowledge should students achieve a level of mastery before moving forward to learn subsequent coursewares? We study the extent to which the prerequisite relation between knowledge concepts in Massive Open Online Courses (MOOCs) can be inferred automatically. In particular, what kinds of information can be leveraged to u...More

Code:

Data:

0
Introduction
  • Mastery learning was first formally proposed by Benjamin Bloom in 1968 (Bloom, 1981), suggesting that students must achieve a level of mastery (e.g., 90% on a knowledge test) in prerequisite knowledge before moving forward to learn subsequent knowledge concepts.
  • Prerequisite relations between knowledge concepts become a cornerstone for designing curriculum in schools and universities.
  • Prerequisite relations essentially can be considered as the dependency among knowledge concepts.
  • It is crucial for people to learn, organize, apply, and generate knowledge (Laurence and Margolis, 1999).
  • The prerequisite knowledge might be “Hidden Markov Model” (in video25 of
Highlights
  • Mastery learning was first formally proposed by Benjamin Bloom in 1968 (Bloom, 1981), suggesting that students must achieve a level of mastery (e.g., 90% on a knowledge test) in prerequisite knowledge before moving forward to learn subsequent knowledge concepts
  • Motivated by the reference distance (RefD) (Liang et al, 2015), we propose three new contextual features, i.e., Video Reference Distance, Sentence Reference Distance and Wikipedia Reference Distance, to infer prerequisite relations in Massive Open Online Courses based on context information from different aspects, which are more general and informative than reference distance and overcome its sparsity problem
  • The experimental results show that our method achieves the state-of-the-art results in the prerequisite relation discovery in Massive Open Online Courses
  • Given a concept pair a, b where a, b ∈ K, we propose the video reference weight (V rw) to quantify how b is referred by videos of a, defined as follows
  • According to the decrement of F1-scores, we find that all the proposed features are useful in predicting prerequisite relations
  • We propose more general features to infer prerequisite relations among concepts, regardless of whether the concept is in Wikipedia or not
Methods
  • The authors first learn appropriate representations for course concepts.
  • Given the course concepts K as input, the authors utilize a Wikipedia corpus to learn semantic representations for concepts in K.
  • The authors' method consists of two steps: (1) entity annotation, and (2) representation learning.
  • The authors first automatically annotate the entities in W to obtain an entity set E and an entity-annotated Wikipedia corpus W = x1 · · · xi · · · xm , where xi corresponds to a word w ∈ W or an entity e ∈ E.
  • The authors use the hyperlinks in Wikipedia articles as annotated entities.
Results
  • The authors apply 5-fold cross validation to evaluate the performance of the proposed method, i.e., testing the method on one fold while training the classifier using the other 4 folds.
  • The authors tried different settings of M and report the results when M =1 and M =10 in Table 2.
  • As for the semantic representation, the authors use the latest publicly available Wikipedia dump 4 and apply the skip-gram model (Mikolov et al, 2013b) to train word embeddings using the Python library gensim 5 with default parameters
Conclusion
  • The authors conducted a new investigation on automatically inferring prerequisite relations among concepts in MOOCs. The authors precisely define the problem and propose several useful features from different aspects, i.e., contextual, structural and semantic features.
  • The authors apply an embeddingbased method that jointly learns the semantic representations of Wikipedia concepts and MOOC concepts to help implement the features.
  • Promising future directions would be to investigate how to utilize user interaction in MOOCs for better prerequisite learning, as well as how deep learning models can be used to automatically learn useful features to help infer prerequisite relations
Summary
  • Introduction:

    Mastery learning was first formally proposed by Benjamin Bloom in 1968 (Bloom, 1981), suggesting that students must achieve a level of mastery (e.g., 90% on a knowledge test) in prerequisite knowledge before moving forward to learn subsequent knowledge concepts.
  • Prerequisite relations between knowledge concepts become a cornerstone for designing curriculum in schools and universities.
  • Prerequisite relations essentially can be considered as the dependency among knowledge concepts.
  • It is crucial for people to learn, organize, apply, and generate knowledge (Laurence and Margolis, 1999).
  • The prerequisite knowledge might be “Hidden Markov Model” (in video25 of
  • Methods:

    The authors first learn appropriate representations for course concepts.
  • Given the course concepts K as input, the authors utilize a Wikipedia corpus to learn semantic representations for concepts in K.
  • The authors' method consists of two steps: (1) entity annotation, and (2) representation learning.
  • The authors first automatically annotate the entities in W to obtain an entity set E and an entity-annotated Wikipedia corpus W = x1 · · · xi · · · xm , where xi corresponds to a word w ∈ W or an entity e ∈ E.
  • The authors use the hyperlinks in Wikipedia articles as annotated entities.
  • Results:

    The authors apply 5-fold cross validation to evaluate the performance of the proposed method, i.e., testing the method on one fold while training the classifier using the other 4 folds.
  • The authors tried different settings of M and report the results when M =1 and M =10 in Table 2.
  • As for the semantic representation, the authors use the latest publicly available Wikipedia dump 4 and apply the skip-gram model (Mikolov et al, 2013b) to train word embeddings using the Python library gensim 5 with default parameters
  • Conclusion:

    The authors conducted a new investigation on automatically inferring prerequisite relations among concepts in MOOCs. The authors precisely define the problem and propose several useful features from different aspects, i.e., contextual, structural and semantic features.
  • The authors apply an embeddingbased method that jointly learns the semantic representations of Wikipedia concepts and MOOC concepts to help implement the features.
  • Promising future directions would be to investigate how to utilize user interaction in MOOCs for better prerequisite learning, as well as how deep learning models can be used to automatically learn useful features to help infer prerequisite relations
Tables
  • Table1: Dataset Statistics prerequisite relations in MOOCs. We created the experimental data sets through a three-stage process
  • Table2: Classification results of the proposed method(%)
  • Table3: Comparison with baselines(%)
  • Table4: Contribution analysis of different features(%)
Download tables as Excel
Related work
  • To the best of our knowledge, there has been no previous work on mining prerequisite relations

    Ignored Feature(s) P R F1 Sr GVrd

    69.6 72.9 71.2(-1.4) 68.8 71.4 70.1(-2.5) Single GSrd Wrd

    67.9 71.4 69.6(-3.0) 70.1 72.1 71.1(-1.5) Apd
Funding
  • This work is supported by 973 Program (No 2014CB340504), NSFC Key Program (No 61533018), Fund of Online Education Research Center, Ministry of Education (No 2016ZD102), Key Technologies Research and Development Program of China (No 2014BAK04B03) and NSFC-NRF (No 61661146007)
Reference
  • Benjamin Samuel Bloom. 198All our children learning: A primer for parents, teachers, and other educators. McGraw-Hill Companies.
    Google ScholarFindings
  • Yixin Cao, Juanzi Li, Xiaofei Guo, Shuanhu Bai, Heng Ji, and Jie Tang. 2015. Name list only? target entity disambiguation in short texts. In Proceedings of EMNLP. pages 654–664.
    Google ScholarLocate open access versionFindings
  • Joseph D. Novak. 1990. Concept mapping: A useful tool for science education. International Journal of Research in Science Teaching 27(10):937C949.
    Google ScholarLocate open access versionFindings
  • Chitu Okoli, Mohamad Mehdi, Mostafa Mesgari, Finn Arup Nielsen, and Arto Lanamaki. 201Wikipedia in the eyes of its beholders: A systematic review of scholarly research on wikipedia readers and readership. International Journal of the American Society for Information Science and Technology (JASIST) 65(12):2381–2403.
    Google ScholarLocate open access versionFindings
  • Mohamed Amir Yosef, Johannes Hoffart, Ilaria Bordino, Marc Spaniol, and Gerhard Weikum. 2011. AIDA: an online tool for accurate disambiguation of named entities in text and tables. Proceedings of the VLDB Endowment (PVLDB) 4(12):1450–1453.
    Google ScholarLocate open access versionFindings
  • Guodong Zhou, Jian Su, and Min Zhang. 200Modeling commonality among related classes in relation extraction. In Proceedings of ACL.
    Google ScholarLocate open access versionFindings
  • Aditya G. Parameswaran, Hector Garcia-Molina, and Anand Rajaraman. 2010. Towards the web of concepts: Extracting concepts from large datasets. Proceedings of the VLDB Endowment (PVLDB) 3(1):566–577.
    Google ScholarLocate open access versionFindings
  • Alan Ritter, Stephen Soderland, and Oren Etzioni. 2009. What is this, anyway: Automatic hypernym discovery. In Proceedings of AAAI. pages 88–93.
    Google ScholarLocate open access versionFindings
  • Jean Michel Rouly, Huzefa Rangwala, and Aditya Johri. 2015. What are we teaching?: Automated evaluation of CS curricula content using topic modeling. In Proceedings of ICER. pages 189–197.
    Google ScholarLocate open access versionFindings
  • Richard Scheines, Elizabeth Silver, and Ilya M. Goldin. 2014. Discovering prerequisite relationships among knowledge components. In Proceedings of EDM. pages 355–356.
    Google ScholarLocate open access versionFindings
  • Nick J Schweitzer. 2008. Wikipedia and psychology: Coverage of concepts and its use by undergraduate students. International Journal of Teaching of Psychology 35(2):81–85.
    Google ScholarLocate open access versionFindings
  • Partha Pratim Talukdar and William W Cohen. 20Crowdsourced comprehension: predicting prerequisite structure in wikipedia. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP. pages 307–315.
    Google ScholarLocate open access versionFindings
  • Annalies Vuong, Tristan Nixon, and Brendon Towle. 2011. A method for finding prerequisites within a curriculum. In Proceedings of EDM. pages 211– 216.
    Google ScholarLocate open access versionFindings
  • Shuting Wang, Alexander Ororbia, Zhaohui Wu, Kyle Williams, Chen Liang, Bart Pursel, and C Lee Giles. 2016. Using prerequisites to extract concept maps fromtextbooks. In Proceedings of CIKM. pages 317–326.
    Google ScholarLocate open access versionFindings
  • Bifan Wei, Jun Liu, Jian Ma, Qinghua Zheng, Wei Zhang, and Boqin Feng. 2012. MOTIF-RE: motifbased hypernym/hyponym relation extraction from wikipedia links. In Proceedings of ICONIP. pages 610–619.
    Google ScholarLocate open access versionFindings
  • Yiming Yang, Hanxiao Liu, Jaime G. Carbonell, and Wanli Ma. 2015. Concept graph learning from educational data. In Proceedings of WSDM. pages 159–168.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments