Machine Learning Basics·Machine learning from data to a responsible model

Overfitting and regularisation

Spot a model that memorises and use regularisation to keep it honest.

Intermediate 14 min read Lesson 7 of 300 Challenge included

What you will be able to do

Compare train and validation scores.
Use cross-validation for a stable estimate.
Try L2 regularisation (Ridge) for correlated features.
More data beats a more complex model surprisingly often.

In plain English

Overfitting is a large gap between training and validation performance. It happens with too much model capacity, too little data or leakage. Regularisation, simpler models and more data are the usual fixes.

regularise.pyPython

from sklearn.linear_model import Ridge
from sklearn.model_selection import cross_val_score

model = Ridge(alpha=1.0)
scores = cross_val_score(model, X, y, cv=5, scoring='neg_mean_absolute_error')
print(-scores.mean())

Your turn

Use cross-validation

Medium

Run 5-fold cross-validation and print the mean score.

PYTHON · starter

cross_val_score(model, X, y, cv=5)

Check your understanding before moving on.

Knowledge checkQuestion 1

What is a sign of overfitting?

Select an answer to continue

Study tools

Make it yours.

Keep a private note for this lesson, then turn the key ideas into flashcards you can revisit.

Sign in to save notes and flashcards.Both are private to your account and stay with your progress.

Finished reading?

Lock in what you just learned.

Mark this lesson complete to save your place and move one step closer to the certificate.

Next: Clustering

Lesson 7 of 300 · 0% of course