Experience Report: How Effective is Automated Program Repair for Industrial Software?

Recent advances in automated program repair (APR) have widely caught the attention of industrial developers as a way of reducing debugging costs. While hundreds of studies have evaluated the effectiveness of APR on open-source software, industrial case studies on APR have been rarely reported; it is still unclear whether APR can work well for industrial software. This paper reports our experience applying a state-of-the-art APR technique, ELIXIR, to large industrial software consisting of 150+ Java projects and 13 years of development histories. It provides lessons learned and recommendations regarding obstacles to the industrial use of current APR: low recall (7.7%), lack of bug-exposing tests (90%), low success rate (10%), among others. We also report the preliminary results of our ongoing improvement of ELIXIR. With some simple enhancements, the success rate of repair has been greatly improved by up to 40%.