Leveraging Crowd for Collecting and Maintaining Educational Resources for Privacy Learning

As the Internet grows in importance, it is vital to develop methods and techniques for educating end-users to improve their awareness of online privacy. Web-based education tools have been proven effective in many domains and have been increasingly adopted by many online professional and educational services. However, the design and development of Web-based education tools for online privacy is still in its early stage. The traditional solutions always involve privacy experts who have sophisticated expertise. However, it is not clear how inspiring and effective these education tools are to general users of varying backgrounds. Furthermore, such involvement can make the tool development costly. In this paper, we discuss our experiences of designing a quiz-based privacy learning tool by leveraging the wisdom of the crowd on Amazon Mechanic Turk for two purposes: (1) designing the quiz questions for the tool; and (2) evaluating and assessing the effectiveness of the tool. Empirical study demonstrates that the crowd can provide high-quality educational materials for privacy learning.