Tree-based methods, or decision tree methods, may be used for two broad types of problem–classification and regression. These methods may be appropriate when there are extensive data, and there is uncertainty about the form in which explanatory variables ought to enter into the model. They may be useful for initial data exploration. Tree-based methods have been especially popular in the data mining community.

Tree-structured classification has a long history in biology, where informal methods of dendrogram construction have been in use for centuries. Social scientists began automating tree-based procedures for classification in the 1940s and 1950s, using methods which are similar to some of the current partitioning methods; see Belson (1959). Venables and Ripley (2002, Chapter 9) give a short survey of the more recent history.

The tree-based regression and classification methodology is radically different from the methods discussed thus far in this book. The theory that underlies the methods of earlier chapters has limited relevance to tree-based methods. The methodology is relatively easy to use and can be applied to a wide class of problems. It is at the same time insensitive to the nuances of particular problems to which it may be applied.

The methodology makes limited use of the ordering of values of continuous or ordinal explanatory variables. In small data sets, it is unlikely to reveal data structure. Its strength is that, in large data sets, it has the potential to reflect relatively complex forms of structure, of a kind that may be hard to detect with conventional regression modeling.