Semi-supervised learning is a machine learning approach that trains a model using a small amount of labelled data together with a large amount of unlabelled data. It sits between supervised and unsupervised learning. The goal is to get close to fully supervised accuracy without paying to label everything.
Key takeaways
- It is useful when labels are expensive but raw data is plentiful.
- Common methods include pseudo-labelling, consistency regularisation and label propagation.
- It relies on the idea that similar data points usually share the same label.
- If early predictions are wrong, errors can spread, so validation is essential.
Semi-supervised learning explained in simple terms
A new employee is shown fifty invoices already sorted into categories by a senior colleague. Then they are handed ten thousand unsorted invoices. Using the fifty as a guide, they sort the easy ones first, learn from those, and gradually handle harder cases. Semi-supervised learning works the same way: a few labelled examples teach the basics, and the model learns more from the structure of the unlabelled data.
Why it matters
In many fields, collecting data is cheap but labelling it is not. Medical scans need specialists to label. Legal documents need lawyers. Speech needs careful transcription. Semi-supervised learning lets teams use all the data they have, not just the small labelled share.
How does semi-supervised learning work?
- Train an initial model on the small labelled set.
- Predict labels for the unlabelled data.
- Keep confident predictions as “pseudo-labels”.
- Retrain on the labelled data plus the pseudo-labelled data.
- Repeat, checking results on a labelled validation set each round.
Main methods
| Method | Idea |
|---|---|
| Self-training / pseudo-labelling | The model labels unlabelled data and learns from its most confident guesses |
| Consistency regularisation | The model should give the same answer when an input is slightly changed, such as a cropped or rotated image |
| Label propagation | Labels spread through a graph from labelled points to nearby similar points |
| Co-training | Two models trained on different views of the data label examples for each other |
| Pre-train then fine-tune | Learn general patterns from unlabelled data first, then fine-tune on labelled data |
The assumptions behind it
Semi-supervised learning works when certain assumptions hold. The smoothness assumption: points close together probably share a label. The cluster assumption: data forms groups, and each group tends to have one label. The manifold assumption: high-dimensional data actually lies on a simpler, lower-dimensional structure. When these assumptions fail, unlabelled data can hurt instead of help.
Semi-supervised vs self-supervised learning
They sound similar but differ. Semi-supervised learning uses some human labels plus unlabelled data. Self-supervised learning uses no human labels at all; it creates training tasks from the data itself, such as predicting a hidden word. In practice, a common modern pattern combines both: self-supervised pre-training, then fine-tuning on a small labelled set.
How to know if it is working
Always compare against a baseline trained only on the labelled data. If adding unlabelled data does not improve results on a held-out labelled test set, it is not helping. Also check performance by class; pseudo-labelling can favour common classes and neglect rare ones.
Limitations
Wrong pseudo-labels can reinforce themselves, a problem called confirmation bias. Imbalanced data can get worse. Methods are sensitive to confidence thresholds. And when unlabelled data comes from a different source than labelled data, gains can disappear.