Machine Learning

The F1 score combines precision and recall into a single number using their harmonic mean, which penalises models that perform well on one measure and badly on the other.

Precision measures what share of a model's positive predictions are correct, while recall measures what share of the real positive cases the model found. The two trade off against each other, and which matters more depends on the cost of each error.

A confusion matrix is a table showing how a classification model's predictions compare with the true labels, breaking results into true positives, false positives, true negatives and false negatives.

Data augmentation is the practice of creating additional training examples by transforming existing data in ways that preserve the label, such as flipping images or paraphrasing sentences.

Data labeling is the process of annotating raw data with the correct answers so supervised models can learn from it, covering everything from tagging images to writing ideal responses for AI assistants.

Data leakage is when information that would not be available at prediction time finds its way into training, producing models that score brilliantly in testing and fail in production.

Model collapse is the progressive degradation that occurs when AI models are trained on data generated by earlier AI models, causing them to lose rare knowledge and converge on bland, narrow outputs.

Early stopping is a regularisation technique that halts training when validation performance stops improving, preventing the model from overfitting the training data while saving compute.

L2 regularization adds a penalty equal to the sum of the squared weights, shrinking all weights smoothly towards zero without eliminating any. It is known as ridge regression in linear models and weight decay in neural networks.

L1 regularization, known as lasso in linear models, adds a penalty equal to the sum of the absolute values of the model's weights, which shrinks some weights to exactly zero and performs automatic feature selection.

Cross-validation is a technique that repeatedly splits data into training and validation parts, rotating which part is held out, so model performance is estimated more reliably than from a single split.

Hyperparameter tuning is the process of systematically searching for the configuration values that make a model perform best on validation data, using methods such as grid search, random search and Bayesian optimisation.

A hyperparameter is a configuration value set before training that controls how a model learns or how complex it can become, such as the learning rate, batch size or number of layers.

Stochastic gradient descent (SGD) updates a model's weights using one example or a small batch at a time rather than the entire dataset, making training fast enough to scale while adding useful noise.

The bias-variance tradeoff describes how a model's error splits into bias from overly simple assumptions and variance from sensitivity to training data, and why reducing one often increases the other.