Skip to content

Who Invented Decision Trees?

CART (Classification and Regression Trees) algorithm is quite popular today and it is also implemented in decision tree objects of Scikit-learn. CART was found by Leo Breiman et al in 1984 and published originally in paper:

Leo Breiman went on to improve on CART model in following years and made many contributions to practical decision tree machine learning implementations. You can find his Berkeley page below where he also shared many of his significant publications related to decision trees:

Breiman was a defendant of algorithmic models and his work as a statistician has been very popular among computer scientists and especially machine learning researchers.

He shares his critical view on abstract route taken in statistics community in his Statistical Science Paper:

There are two cultures in the use of statistical modeling to reach conclusions from data. One assumes that the data are generated by a given stochastic data model. The other uses algorithmic models and treats the data mechanism as unknown. 

The statistical community has been committed to the almost exclusive use of data models. This commitment has led to irrelevant theory, questionable conclusions, and has kept statisticians from working on a large range of interesting current problems.

-Leo Breiman, 2001

Thanks to his curios and critical mind today we have not only decision trees but also bagging implementations and random forests.