Reinforcement Learning Guided Software Debloating∗

Modern software engineering practices encourage software bloat. Libraries provide more APIs than any one client needs; programs have many command line and configuration options, with most never used by the user and remaining set to their default values. The result is software systems that are bloated, slow, and, most importantly, vulnerable to attacks. Software debloating is an emerging technique to address this problem by automatically trimming (or specializing) a program to a user-defined execution environment (e.g., command line options, configuration files, common usage scenarios, etc.). Since deciding on what to specialize is computationally expensive, current state-of-the-art software debloaters rely on hand-crafted heuristics. This is problematic because these heuristics are not always effective, and, more importantly, difficult to adjust to different debloating metrics. In this paper, we propose a reinforcement learning based approach to automatically learn a good debloating policy. We have implemented the approach as a tool called DEEPOCCAM, and evaluated it on debloating several C programs. Our results show that the technique outperforms the base-line policy of a state-of-the-art debloater OCCAM, while being easily adaptable to a variety of debloating metrics.