One of the objectives of ensemble techniques is to improve model accuracy by driving down the variance without affecting too much the bias. In this note, we consider bagging trees. Bagging trees is an ensemble technique which consists in combining several regression trees fitted on different bootstrap samples of the training set. We demonstrate that bagging trees performs better than one of its constituent trees in the sense of the expected generalization error. Moreover, we show through an example that bagging trees outperforms not only one of its constituent tree but also the best decision tree built on the entire training set.

Keywords: Bagging trees, regression tree, generalization error, Poisson deviance loss.