State-of-the-BART: Simple Bayesian Tree Algorithms for Prediction and Causal Inference

Bayesian Additive Regression Trees (BART) (Chipman et al. 2010) and Bayesian Causal Forests (BCF) (Hahn et al. 2017) are state-of-the-art machine learning algorithms for prediction and individual treatment effect estimation. These methods involve averaging predictions from sum-of-tree models, typically drawn using Monte Carlo Markov Chain methods. This paper introduces conceptually and computationally simple alternatives to MCMC implementations of BART, which can exhibit comparable performance. An importance sampling based implementation of BART (BART-IS) builds on the ideas of Hernández et al. (2018) and Quadrianto & Ghahramani (2014). Unlike most BART implementations, BART-IS has a data independent model prior. This paper also contains an extension to treatment effect estimation, BCF-IS. In addition, I describe Bayesian Causal Forests using Bayesian Model Averaging (BCF-BMA), an implementation of BCF (Hahn et al. 2017) that extends an improved implementation of BART-BMA (Hernández et al. 2018) to treatment effect estimation. 1 ∗Faculty of Economics, University of Cambridge, Cambridge CB3 9DD, UK. Email: epo21@cam.ac.uk. My thanks are due to Melvyn Weeks, Alexey Onatskiy, Belinda Hernandez, Andrew Parnell, and seminar participants at the Maynooth University Hamilton Institute. The latest version of this paper is available at https://eoghanoneill.com/research/ . 1R packages implemented in C++ for the methods described in this paper are available at https://github.com/EoghanONeill .