Detecting JavaScript Transpiler Bugs with Grammar-guided Mutation

JavaScript (JS) transpilers translate JS programs from a higher grammar standard to a lower one, which are widely used to ensure the compatibility of JS features in software (e.g., browsers). However, JS transpilers can have bugs that lead to unintended behavior in the translated JS programs. Existing JS program generation approaches could not test JS transpilers effectively since it is hard to generate a large number of valid JS programs in specific grammar standards. In this paper, we propose TransFuzz, a grammar-guided mutation approach to find JS transpiler bugs.The key insight of TransFuzz is to generate syntax-specific JS programs by mutating the abstract syntax trees (ASTs) of JS programs with the guidance of the specific grammar. First, Trans- Fuzz parses JS programs collected from open-source platforms into ASTs to obtain subtrees and leaf nodes containing specific JS syntax. Then, a grammar-guided approach is developed in TransFuzz to mutate the ASTs of the given JS programs guided by different versions of JS grammar standards. In addition, mutation operations could introduce grammatical errors. To improve the correctness of the mutated ASTs, TransFuzz develops heuristic-based correction rules to correct reference errors, type errors, and syntax errors in the mutated ASTs. After correction, the mutated ASTs are converted to the corresponding JS programs. Finally, based on differential testing, TransFuzz utilizes the generated JS programs to detect JS transpiler bugs.Our evaluation shows that TransFuzz significantly outperforms existing JS program generation approaches by triggering 47.82%-385.71% more JS transpiler bugs. Within ten months, we have reported 73 bugs on two popular JS transpilers babel and swc, of which 58 have been confirmed.