Introduction

A method for making decisions that is both primitive and surprisingly effective is the single decision tree. It has a sound mathematical basis showing how well it does with large enough samples, and its extreme simplicity has led to a wide range of practical and effective variants. And generating many single trees lies at the heart of the Random Forests and Random Jungles procedures, which have been applied with increasing regularity and satisfying results.

Many good texts are available for decision trees, including Breiman et al. (1993), Hastie et al. (2009), Berk (2008). Mastering the details of tree growth and management is an excellent way to understand the activities of learning machines generally.

Let's begin our tree discussion with something near the end of the story: given a single tree, how does it make a decision?

Dropping down trees

Consider Figure 6.1. This is a single decision tree, derived using the at-risk boundaries we started with in Chapter 4. Let's assume for the moment that these at-risk determinations were constructed from the data, and that the three boundary lines in this example were estimated from the data. Now, for making predictions on a new subject the tree would work this way.