Overfitting happens when a machine learning model learns its training data too closely, including random noise and quirks, so it performs very well on that data but poorly on new data. The model has memorised examples instead of learning general patterns. It is one of the most common problems in machine learning.
Key takeaways
- The classic sign is high training accuracy with much lower validation or test accuracy.
- Causes include models that are too complex, too little data and noisy features.
- Fixes include more data, regularisation, simpler models, dropout and early stopping.
- Overfitting is linked to high model variance.
Overfitting explained in simple terms
A student memorises the exact answers to last year’s exam paper without understanding the subject. If this year’s paper is identical, they score 100%. If the questions are even slightly different, they fail. An overfit model is that student. It has learned the specific examples, not the underlying rules.
How to spot overfitting
| Signal | What it suggests |
|---|---|
| Training score very high, validation score much lower | Classic overfitting |
| Validation loss rises while training loss keeps falling | Training has gone on too long |
| Performance changes a lot with small data changes | High variance |
| Model relies on odd or irrelevant features | It has learned noise or shortcuts |
| Great test score, poor production results | Possible leakage or overfitting to test data |
What causes overfitting?
- Model too complex for the amount of data, such as a very deep decision tree or huge neural network on a small dataset.
- Too little training data to show the true pattern.
- Noisy or wrong labels that the model learns as if they were real.
- Too many features, especially irrelevant ones.
- Training for too long, particularly in neural networks.
- Over-tuning on the validation set, which overfits your decisions to that data.
How to prevent overfitting
| Technique | How it helps |
|---|---|
| More training data | Makes true patterns clearer than noise |
| Data augmentation | Creates varied examples, such as rotated or cropped images |
| Regularisation (L1, L2) | Penalises large weights, pushing the model toward simpler solutions |
| Simpler model | Fewer parameters, shallower trees |
| Dropout | Randomly switches off neurons during training so the network cannot rely on any one path |
| Early stopping | Stops training when validation loss stops improving |
| Cross-validation | Gives a more reliable estimate of generalisation |
| Feature selection | Removes noisy or irrelevant inputs |
| Ensembles | Averaging many models, as in random forests, reduces variance |
Overfitting vs underfitting
Overfitting is learning too much detail. Underfitting is learning too little. An underfit model does poorly on both training and validation data. The goal is a model in between: complex enough to capture real patterns, simple enough to ignore noise. This balance is known as the bias-variance trade-off, covered in model bias and model variance.
Overfitting in deep learning
Modern neural networks often have more parameters than training examples, yet can still generalise well, especially with regularisation, data augmentation and large datasets. Researchers have also observed “double descent”, where test error can fall again as models grow very large. Overfitting is still real in deep learning, but the relationship between size and overfitting is more complex than classic theory suggested.
Why it matters in business
An overfit model looks impressive in development and disappoints in production. That wastes budget and damages trust in AI projects. Using clean data splits, honest validation and simple baselines is the best protection.