A GRU (gated recurrent unit) is a type of recurrent neural network that uses two gates, an update gate and a reset gate, to control how much past information to keep and how much new information to add. It was introduced in 2014 by Kyunghyun Cho and colleagues as a simpler alternative to LSTM, and often reaches similar accuracy with fewer parameters.
Key takeaways
- A GRU has two gates; an LSTM has three.
- It has no separate cell state: the hidden state itself carries memory.
- Fewer parameters usually means faster training and less data needed.
- It solves the same long-memory problem as LSTM, the vanishing gradient in plain RNNs.
- Neither consistently beats the other; test both on your data.
GRU explained in simple terms
Imagine a running summary of a story that you update after every sentence. For each new sentence you ask two questions. First, how much of my old summary is relevant to understanding this sentence? (reset gate). Second, how much should I replace my summary with this new information, versus keep it as it is? (update gate). A GRU learns to answer these questions automatically at every step.
The two gates
| Gate | What it controls | When it is near 1 | When it is near 0 |
|---|---|---|---|
| Update gate | Balance between old memory and new information | Mostly adopt the new candidate state | Keep the old state largely unchanged |
| Reset gate | How much past memory to use when forming new information | Use past context fully | Ignore past context, start fresh |
The update gate effectively does the combined job of the LSTM’s forget and input gates. This merging is the main reason GRUs are smaller.
How a GRU step works
- Read the new input and the previous hidden state.
- Compute the reset gate and the update gate (each outputs values between 0 and 1).
- Build a candidate new state using the input and a reset-filtered version of the old state.
- Blend the old state and the candidate using the update gate.
- Output the blended result as the new hidden state.
A worked example: a sensor that changes mode
Suppose a GRU reads temperature readings from a machine every minute. For hours the readings drift slowly; the update gate stays low, so memory of the long-term trend is preserved. Then the machine switches to a new mode and readings jump. The reset gate can drop low, so the model stops leaning on the old pattern, and the update gate rises to adopt the new state quickly. This ability to hold steady or reset is what makes gated networks good at real-world sequences.
GRU vs LSTM
| Aspect | GRU | LSTM |
|---|---|---|
| Gates | 2 (update, reset) | 3 (forget, input, output) |
| Memory | Hidden state only | Separate cell state plus hidden state |
| Parameters per unit | Fewer (roughly three quarters of an LSTM’s) | More |
| Training speed | Usually faster | Usually slower |
| Small datasets | Often a good choice | Can overfit more easily |
| Very long, complex dependencies | Sometimes slightly weaker | Sometimes slightly stronger |
Empirical comparisons, including a 2014 study by Junyoung Chung and colleagues, found the two often perform similarly, with the winner depending on the task.
When to use a GRU
- You need a compact recurrent model for mobile or embedded devices.
- Your dataset is modest and you want fewer parameters to reduce overfitting.
- You are forecasting or classifying time series, logs or sensor streams.
- Training time is limited.
For large-scale language tasks, transformers have largely replaced both GRUs and LSTMs.
Limitations
- Still processes one step at a time, so training cannot be parallelised across the sequence.
- Can be less expressive than LSTM on some complex tasks.
- Less effective than attention-based models for very long contexts.
- Performance differences versus LSTM are data-dependent, so both usually need testing.