Automatic Feature Generation for Setting Compilers Heuristics

Heuristics in compilers are often designed by manually analyzing sample pro- grams. Recent advances have successfully applied machine learning to automati- cally generate heuristics. The typical format of these approaches reduces the input loops, functions or programs to a finite vector of features. A machine learning algorithm then learns a mapping from these features to the desired heuristic pa- rameters. Choosing the right features is important and requires expert knowledge since no machine learning tool will work well with poorly chosen features. This paper introduces a novel mechanism to generate features. Grammars de- scribing languages of features are defined and from these grammars sentences are randomly produced. The features are then evaluated over input data and computed values are given to machine learning tools. We propose the construction of domain specific feature languages for different purposes in different parts of the compiler. Using these feature languages, com- plex, machine generated features are extracted from program code. Using our observation that some functions can benefit from setting different compiler op- tions, while others cannot, we demonstrate the use of a decision tree classifier to automatically identify the former using the automatically generated features. We show that our method outperform human generated features on problems of loop unrolling and phase ordering, achieving a statistically significant decrease in run-time compared to programs compiled using GCC's heuristics.