ScriptButler serves an Empirical Study of PuzzleScript Analyzing the Expressive Power of a Game DSL through Source Code Analysis

Automated Game Design (AGD) empowers game designers with languages and tools that automate game design processes. Domain-Specific Languages (DSLs) promise to deliver an expressive means for rapidly prototyping and fine-tuning interaction mechanisms that support rich emergent player experiences. However, despite the growing number of studies that center around languages for games and play, few prototypes are ever thoroughly validated and evaluated in practice. As a result, it is not yet well understood what the costs, benefits and limitations of DSL formalisms are. To find out, we investigate to what extent rules, affordances and play can be related by means of source code analysis. We study PuzzleScript, a language and online game engine with an active user community. We reverse engineer PuzzleScript's design and propose ScriptButler, a novel tool prototype and engine for its analysis. To validate our approach, we conduct an empirical study on the quality of the source code by performing an analysis on a curated collection of 95 games. Our results show that ScriptButler can identify bugs and helps relate PuzzleScript rules to game qualities.