Backpropagation is the algorithm neural networks use to learn. After a prediction, it works backwards from the error through each layer, using the chain rule of calculus to calculate how much every weight contributed, so the weights can be adjusted.
A diffusion model is a generative AI model that learns to create data by reversing a noising process: it starts from pure random noise and removes noise step by step until a clear image, audio clip or video emerges.
A GAN (generative adversarial network) is a generative model made of two competing neural networks: a generator that creates fake data and a discriminator that tries to tell fake from real. As they compete, the generator learns to produce highly realistic output.
An autoencoder is a neural network trained to compress its input into a small representation and then reconstruct the original from it. By learning what to keep, it discovers the most important features of the data without labels.
A decoder is the part of a neural network that turns an internal numeric representation into output, such as a sentence, an image or audio. In language models, decoders generate text one token at a time, each time using what has been generated so far.
An encoder is the part of a neural network that converts raw input, such as text, images or audio, into a compact numeric representation that captures its meaning. Other components then use this representation to classify, search or generate.
A GRU (gated recurrent unit) is a simpler variant of the LSTM recurrent network. It uses two gates, reset and update, to control memory across a sequence, often matching LSTM accuracy with fewer parameters and faster training.
LSTM (long short-term memory) is a type of recurrent neural network that uses a memory cell and three gates to decide what to remember, what to forget and what to output, allowing it to learn patterns across long sequences.
An AI model checkpoint is a saved snapshot of a model's learned weights, and often its training state, at a specific point in training. Checkpoints let teams resume training, compare versions, share models and deploy them.
A system prompt is a set of instructions given to an AI model, usually by the developer, that defines its role, rules, tone and boundaries for an entire conversation, separate from the user's individual messages.
Temperature is a setting that controls how random or predictable a language model's output is. Low temperature makes the model choose the most likely words; high temperature spreads probability out, producing more varied and creative text.
LLM inference is the process of running a trained large language model to generate a response: processing the prompt, then producing output one token at a time. Its speed and cost depend on model size, context length, hardware and serving techniques.
LoRA (low-rank adaptation) is a parameter-efficient fine-tuning method that freezes a model's original weights and trains two small matrices whose product represents the weight update, dramatically reducing the parameters and memory needed.
A multimodal LLM is a large language model that can process more than text, such as images, audio or video, usually by converting them into token-like representations the language model can reason over alongside text.
Structured output is when an AI model returns its response in a predefined, machine-readable format, such as JSON matching a schema, instead of free text, so software can reliably use the result.