Chapter 2: Linear Regression for Regression, Logistic Regression for Classification and Statistical Forecasting
This chapter covers linear regression for regression problems and logistic regression for classification problems, closing with statistical forecasting applied to environmental time series.
Classification vs. regression, cost functions, gradient descent, and the metrics used to evaluate a classifier.
Tuning a k-nearest-neighbours classifier on MNIST digits with grid search, then a random forest and a support-vector classifier on the Titanic dataset.
Implementing logistic regression from scratch — the logistic function, log loss, and gradient descent — on Palmer penguin bill measurements.
Regression as a forecasting tool: feature selection, postprocessing, prediction intervals, and model output statistics.
Fitting a linear regression to real pressure and temperature readings, then a logistic regression to classify El Niño years.
Resources¶
Textbooks this chapter’s exercises adapt¶
Hands-On Machine Learning with Scikit-Learn — Aurélien Géron; the source of the classification and training-models exercises. (2.2, 2.3)
Statistical Methods in the Atmospheric Sciences — Wilks; the source of the pressure/temperature data used in the statistical-forecasting exercise. (2.5)
Scikit-learn¶
Getting started — the estimator interface, fit/predict, pipelines. (2.1–2.5)
Model selection: GridSearchCV — hyperparameter search with cross-validation. (2.2)
Metrics and scoring — accuracy, log loss, and the other classification metrics used throughout. (2.1, 2.2)
Datasets¶
palmerpenguins — Horst, Hill & Gorman (2020); the real dataset behind the from-scratch logistic regression. (2.3)
MNIST and the Titanic dataset — the two real datasets behind the classification exercises. (2.2)