Repetition penalty is a decoding setting that reduces the probability of tokens that have already appeared in the text, so the model is less likely to repeat itself. It was introduced with the CTRL model in 2019 and is widely used in open-source generation tools. It applies a multiplicative adjustment to the logits of tokens already seen.
Key takeaways
- A value of 1.0 means no penalty; higher values penalise repetition more.
- Typical useful values are 1.05 to 1.2.
- It applies to any token that has appeared, regardless of how often.
- Too high a value harms fluency, since some repetition is natural.
- It differs from OpenAI’s frequency and presence penalties, which are additive.
Repetition penalty explained in simple terms
Imagine an editor who crosses out any word you have already used, making you reach for alternatives. Used lightly, your writing becomes more varied. Used heavily, you can no longer write “the” or “and” naturally, and the text falls apart. That balance is what the penalty value controls.
How it works
For each token that has already appeared, its logit is divided by the penalty value if the logit is positive, or multiplied by it if negative. Both cases push the token’s probability down. The adjusted logits then go through softmax and sampling as usual.
Worked example with a penalty of 1.2. The token ” however” has already appeared and has a logit of 3.0.
- Adjusted logit: 3.0 ÷ 1.2 = 2.5.
- After softmax, its probability falls relative to unused tokens, making a repeat less likely but not impossible.
Choosing a value
| Value | Effect |
|---|---|
| 1.0 | No penalty |
| 1.05–1.1 | Light touch; usually safe |
| 1.1–1.2 | Common range for reducing loops |
| 1.3+ | Often degrades grammar and coherence |
Why repetition happens at all
Language models trained with next-token prediction tend to assign rising probability to text they have already produced, especially with greedy decoding or low temperature. This can create loops where a phrase repeats indefinitely. Penalties are one fix; better sampling with top-p is another.
Side effects to watch
- Legitimate repetition suffers: lists, names, technical terms and code identifiers.
- In code generation, penalising repeated variable names can break output.
- In structured output, repeated JSON keys are necessary, so penalties should usually be off.
- Long outputs penalise more tokens over time, which can cause drift.
Where you will find it
Repetition penalty is standard in open-source stacks such as Hugging Face Transformers, llama.cpp and vLLM. Commercial APIs more often expose frequency and presence penalties instead. Some tools also offer a “no repeat n-gram size” setting, which blocks exact repeats of a phrase of a given length.
Limitations
- A blunt instrument: it does not know which repetition is desirable.
- Penalty strength that works for prose may break code.
- Interacts with temperature and top-p, complicating tuning.
- Does not fix the underlying cause of degenerate output.