Tree-Based Algorithms
Before deep networks took over perception, and still today on most tabular data, tree-based models are the tools that win. This section builds them from the ground up: a single decision tree, the bias-variance lens that explains when it fails, and the two great ideas for combining trees, bagging (random forests) and boosting (AdaBoost, gradient boosting).
Every lesson is taught on three pillars at once: the math that makes it exact, an interactive visual that makes it obvious, and a runnable Python playground (scikit-learn, in your browser) so you build it yourself.
Prerequisites
Foundations of Regression
Linear and logistic regression, gradient descent, decision boundaries
Performance Metrics
Confusion matrix, precision/recall, MSE/RMSE, R2, held-out evaluation
Python, NumPy & scikit-learn
Array operations, fitting an sklearn estimator, reading a plot
Lessons
Decision Trees
Split feature space with the most informative questions
Overfitting, Bias-Variance & Cross-Validation
Diagnose the two ways a model fails, and measure honestly
Random Forests & Bagging
Average many decorrelated trees to crush variance
Boosting: AdaBoost & Gradient Boosting
Fix mistakes sequentially to crush bias
Unlocks
Deep Neural Networks
Trade hand-built splits for learned features and universal function approximation
Gradient Boosting in practice
XGBoost, LightGBM, and CatBoost — the libraries that win tabular competitions