AI Glossaries

An open-weight model is an AI model whose trained weights are published so anyone can download, run and often fine-tune it. It differs from fully open-source AI, which also shares training code and data details.

Model distillation, or knowledge distillation, trains a smaller 'student' model to imitate a larger 'teacher' model. The student learns from the teacher's outputs and becomes cheaper and faster to run while keeping much of the teacher's ability.

Model quantization reduces the precision of a model's numbers, for example from 16-bit to 8-bit or 4-bit, so it uses less memory and runs faster, usually with only a small loss in quality.

DPO (direct preference optimization) is a method for aligning language models with human preferences. It trains directly on pairs of preferred and rejected responses, avoiding the separate reward model and reinforcement learning loop used in RLHF.

RLHF (reinforcement learning from human feedback) is a training method where people compare AI responses, a reward model learns their preferences, and the language model is then optimised with reinforcement learning to produce responses people prefer.

Instruction tuning is a fine-tuning step that trains a pretrained language model on many examples of instructions paired with good responses, turning a text-continuing base model into an assistant that follows requests.

Pretraining is the first and largest stage of training a foundation model, in which the model learns general knowledge and language patterns from vast amounts of unlabelled data, usually by predicting the next token.

A language model is an AI model that estimates how likely a sequence of words is, usually by predicting the next word or token. Language models range from simple word-count statistics to today's large neural networks.

A vector embedding is a list of numbers generated by an embedding model to represent a piece of content, such as a sentence, document or image, so that computers can measure how similar two pieces of content are by comparing their vectors.

Parameter-efficient fine-tuning (PEFT) is a family of methods that adapt large pretrained models by training only a small fraction of parameters, keeping most of the original model frozen, which cuts memory, cost and storage.

Fine-tuning is the process of taking a pretrained AI model and training it further on a smaller, specific dataset so it performs better on a particular task, domain or style.

Supervised fine-tuning (SFT) is the process of further training a pretrained model on labelled examples, usually prompts paired with ideal responses, so it learns specific behaviours, formats or tasks.

A reasoning model is a large language model trained to work through problems step by step, generating internal 'thinking' before its final answer. It uses extra computation at inference time to improve accuracy on maths, coding, science and planning.