A Survey on Automated Code Evaluation Systems and Their Resources for Code Analysis.

The automated code evaluation system is designed to reliably evaluate user-submitted code. Code is first compiled and then tested on a homogeneous surface using defined input and output test cases. Automated code evaluation systems are gaining popularity due to their wide range of applications and valuable accumulated resources. The success of machine learning techniques emboldens researchers to use them for source code analysis tasks, and a large number of real-life solution codes from automated evaluation systems adds significant value. In this paper, we review the state-of-the-art of automated code evaluation systems and their resources for code analysis tasks using machine learning. We classify these code evaluation systems into several categories, including programming contests, programming learning, recruitment, online compilers, and additional modules of other systems. We research the datasets available in these systems for code analysis. Moreover, we summarize the machine learning-based code assessment tasks, including error detection, code comprehension, review, search and representation, refactoring, and repair using these datasets.