LARC: Language annotated Abstraction and Reasoning Corpus

Sam Acquaviva,Yewen Pu,Maxwell Nye,Catherine Wong, Michael Tessler, Josh Tenenbaum

Proceedings of the Annual Meeting of the Cognitive Science Society(2021)

引用 0|浏览0
Author(s): Acquaviva, Sam; Pu, Yewen; Nye, Maxwell; Wong, Catherine; Tessler, Michael Henry; Tenenbaum, Josh | Abstract: The Abstraction and Reasoning Challenge (ARC) is a set of tasks where one must induce a program from a few given input-output examples, and apply it to a new input. Although humans can easily solve most tasks, ARC is challenging for state-of-the-art algorithms. We hypothesize that humans use intuitive program induction, and interpret ARC tasks by constructing programs'' in language. We experimentally study programs'' by formulating a two-player game: A participant solves and then communicates the program to another participant using natural language; the second participant must solve the task using the description alone. We find at least 361 out of 400 tasks can be solved from a natural language description, demonstrating that natural language is sufficient in transmitting these natural programs. We compared the natural language programs to computer programs constructed using two separate state-of-the-art program synthesis approaches, and conducted a study on leveraging natural language annotations to improve performance of program-synthesis tools.
AI 理解论文