A hidden layer is any layer of neurons that sits between a neural network’s input layer and its output layer. It is called “hidden” because its values are not directly given or observed; they are internal representations the network learns for itself. Hidden layers do most of the real work, transforming raw input into features that make the final prediction possible.
Key takeaways
- Networks have one input layer, one output layer and any number of hidden layers.
- Each hidden layer learns progressively more abstract features.
- Depth is the number of hidden layers; width is the number of neurons per layer.
- Networks with many hidden layers are the basis of deep learning.
- More layers add capacity but also cost and the risk of overfitting.
A hidden layer explained in simple terms
Think of an assembly line turning raw materials into a car. The input is steel, glass and rubber. The output is the finished car. In between are workstations nobody outside the factory sees: one makes panels, one builds the engine, one assembles the chassis. Hidden layers are those workstations. Each takes what the previous one produced and turns it into something more useful for the final result.
What hidden layers learn
In an image model built with CNN layers, researchers have visualised this progression:
- First hidden layers respond to edges, colours and simple textures.
- Middle layers respond to corners, curves and repeated patterns.
- Deeper layers respond to object parts, such as eyes, wheels or letters.
- The final hidden layers represent whole concepts that make the output easy.
In language models, deeper layers tend to capture grammar, then meaning, then more abstract relationships, though the picture is less clean than in images.
How a hidden layer computes
Each neuron in a hidden layer:
- Receives all outputs from the previous layer (in a fully connected layer).
- Multiplies each by a weight and adds a bias.
- Applies an activation function such as ReLU.
- Passes its output to every neuron in the next layer.
A worked example: solving XOR
XOR outputs 1 when exactly one of two inputs is 1. A single neuron cannot solve it. Add one hidden layer with two neurons:
- Hidden neuron A learns “at least one input is 1” (OR).
- Hidden neuron B learns “both inputs are 1” (AND).
- The output neuron learns “A is on and B is off”.
| Input | A (OR) | B (AND) | Output (A and not B) |
|---|---|---|---|
| 0, 0 | 0 | 0 | 0 |
| 0, 1 | 1 | 0 | 1 |
| 1, 0 | 1 | 0 | 1 |
| 1, 1 | 1 | 1 | 0 |
The hidden layer turned an impossible problem into an easy one by creating new features. That is the essence of what hidden layers do.
Depth vs width
| Aspect | More depth (layers) | More width (neurons per layer) |
|---|---|---|
| Effect | Builds hierarchical, layered features | Captures more features at one level |
| Efficiency | Often represents complex patterns with fewer total neurons | Can need very many neurons for complex patterns |
| Training | Harder: vanishing gradients, needs residual connections and normalisation | Easier to train, but more parameters per layer |
The universal approximation theorem (George Cybenko, 1989; Kurt Hornik, 1991) shows that even one wide enough hidden layer can approximate any continuous function. In practice, deeper networks usually learn complex tasks far more efficiently, which is why modern models have dozens or hundreds of layers.
How many hidden layers should you use?
- Simple tabular problems: often 1–3 hidden layers, or a tree-based model instead.
- Images and audio: deep CNNs or transformers with many layers, usually starting from pre-trained models.
- Language: large language models have dozens of transformer layers or more.
- General rule: start small, increase depth or width while the validation score improves, and stop when it plateaus.
Types of hidden layers
- Dense (fully connected): every neuron connects to every neuron in the previous layer.
- Convolutional: neurons look at small local regions (CNNs).
- Recurrent: carry state across time steps (RNN, LSTM, GRU).
- Attention / transformer blocks: relate every position to every other position.
- Normalisation and dropout layers: support training rather than learning features directly.
Limitations
- More layers increase compute, memory and training time.
- Very deep networks are hard to train without special techniques.
- Hidden representations are difficult to interpret.
- Excess capacity can cause overfitting on small datasets.