A variational autoencoder (VAE) is a generative model that encodes data into a probability distribution rather than a fixed point, producing a smooth latent space from which new, realistic samples can be generated.
Backpropagation is the algorithm neural networks use to learn. After a prediction, it works backwards from the error through each layer, using the chain rule of calculus to calculate how much every weight contributed, so the weights can be adjusted.
A diffusion model is a generative AI model that learns to create data by reversing a noising process: it starts from pure random noise and removes noise step by step until a clear image, audio clip or video emerges.
A GAN (generative adversarial network) is a generative model made of two competing neural networks: a generator that creates fake data and a discriminator that tries to tell fake from real. As they compete, the generator learns to produce highly realistic output.
An autoencoder is a neural network trained to compress its input into a small representation and then reconstruct the original from it. By learning what to keep, it discovers the most important features of the data without labels.
A decoder is the part of a neural network that turns an internal numeric representation into output, such as a sentence, an image or audio. In language models, decoders generate text one token at a time, each time using what has been generated so far.
An encoder is the part of a neural network that converts raw input, such as text, images or audio, into a compact numeric representation that captures its meaning. Other components then use this representation to classify, search or generate.
A GRU (gated recurrent unit) is a simpler variant of the LSTM recurrent network. It uses two gates, reset and update, to control memory across a sequence, often matching LSTM accuracy with fewer parameters and faster training.
LSTM (long short-term memory) is a type of recurrent neural network that uses a memory cell and three gates to decide what to remember, what to forget and what to output, allowing it to learn patterns across long sequences.
In a neural network, a bias is a learned number added to a neuron's weighted sum before the activation function. It shifts the neuron's output up or down, letting the network fit patterns that do not pass through zero.
A weight is a learned number in a neural network that controls how strongly one input or neuron influences the next. Training is the process of adjusting millions or billions of weights so the network's predictions become accurate.
A hidden layer is any layer of neurons between a neural network's input and output layers. Hidden layers transform the input into increasingly useful internal features, and stacking many of them is what makes a network deep.
An activation function is a mathematical function applied to a neuron's weighted sum that decides its output. It adds non-linearity, which lets neural networks learn complex patterns instead of only straight-line relationships.
A recurrent neural network (RNN) is a type of neural network designed for sequences such as text, speech and time series. It processes one step at a time and carries a hidden state, a kind of short-term memory, from each step to the next.
A convolutional neural network (CNN) is a type of deep learning model designed for grid-like data such as images. It slides small learned filters across the input to detect patterns like edges, textures and shapes, then combines them to recognise objects.