Classification in machine learning is a supervised learning task in which a model learns to assign each input to one of several predefined categories, called classes. The model studies labelled examples, learns a decision boundary that separates the classes, and then predicts the class of new inputs, usually with a probability for each class.
This page covers the technical side: how classifiers learn, which algorithms and metrics to use and how thresholds work. For a plain-English overview of business uses, see What is AI Classification?
Key takeaways
- Classification is one of the two core supervised learning tasks, alongside regression.
- Classifiers usually output probabilities; a threshold turns them into class decisions.
- Cross-entropy is the most common training loss.
- Precision, recall, F1 and ROC-AUC are more informative than accuracy on imbalanced data.
How does a classifier learn?
Picture customers plotted on a chart by age and monthly spend, coloured by whether they churned. A classifier tries to draw a line, or a more complex shape, that separates the colours. That line is the decision boundary. Simple models draw straight boundaries; flexible models draw curved ones. Too simple and the model underfits. Too flexible and it overfits, tracing around individual points.
During training, the model predicts a probability for each class. Cross-entropy loss measures how far those probabilities are from the true labels, heavily penalising confident wrong answers. An optimiser such as gradient descent adjusts the model to reduce the loss.
Types of classification
| Type | Setup | Output layer / approach |
|---|---|---|
| Binary | Two classes | One probability, often via a sigmoid function |
| Multi-class | One of three or more classes | Probabilities across all classes via softmax |
| Multi-label | Several classes at once | Independent probability per label |
| Imbalanced | One class is rare | Class weights, resampling, threshold tuning |
Common algorithms and when to use them
| Algorithm | Strengths | Best for |
|---|---|---|
| Logistic regression | Fast, explainable, good baseline | Regulated or simple problems |
| Decision tree | Easy to visualise | Explaining rules to stakeholders |
| Random forest | Robust, little tuning | General tabular data |
| Gradient boosting (XGBoost, LightGBM, CatBoost) | Often top accuracy on tables | Fraud, credit, churn |
| Support vector machine | Strong on smaller, high-dimensional data | Text features, small datasets |
| k-nearest neighbours | Simple, no training phase | Small datasets, baselines |
| Neural networks | Learn complex patterns | Images, audio, text |
Evaluation metrics
- Confusion matrix: counts true positives, false positives, true negatives and false negatives.
- Precision = TP / (TP + FP): how trustworthy positive predictions are.
- Recall = TP / (TP + FN): how many real positives were caught.
- F1 score: the harmonic mean of precision and recall.
- ROC-AUC: how well the model ranks positives above negatives across all thresholds.
- PR-AUC: more informative than ROC-AUC when positives are rare.
- Log loss: rewards well-calibrated probabilities.
Choosing a threshold
Most classifiers predict “positive” when the probability is above 0.5 by default. That is rarely optimal. Lowering the threshold catches more positives (higher recall) but raises false alarms (lower precision). Pick the threshold using the validation set and the real cost of each error type.
Limitations
Classifiers depend on label quality and can only predict classes they have seen. Imbalanced data can hide poor performance behind high accuracy. Probabilities may be poorly calibrated. And models can pick up spurious shortcuts, such as a watermark that happens to appear in one class of images.