Software Crowdsourcing Design: An Experiment on the Relationship Between Task Design and Crowdsourcing Performance.

Software crowdsourcing platforms allow for wide task accessibility and self-selection, which require participants to interact with a wide range of task options. This increased accessibility enables crowd workers to freely choose tasks based on their skills, experience and interests. However, the inconsistencies in crowd workers’ comprehension and interpretation of the designs of different tasks and the platform interaction strategies may introduce task uncertainty into the selection process. This may result in low performance, which can diminish the quality of the crowdsourcing platform. Previous studies show that the quality of the outcome of these systems depends on different design parameters in both social and technical contexts. Therefore, through the lens of Socio-Technical Systems, our experimental study quantifies and compares 30 participants’ behavior and performance throughout their interaction and selection of different software tasks via an online-based tool named “SoftCrowd,” using different design strategies and task features. Results showed a significant relationship between task submission rate and platform design, including interaction strategies, task ambiguity and task instruction design. These findings suggest that our proposed design factors may have unanticipated effects on the quality of crowdsourcing, calling for more studies of crowdsourcing design to inform better strategies for various aspects of platform design.