Frequency penalty is a decoding setting that reduces a token’s probability in proportion to how many times it has already appeared in the generated text. The more often a token is used, the stronger the penalty becomes. It is exposed by several major LLM APIs, usually on a scale from −2.0 to 2.0, with 0 meaning no penalty.
Key takeaways
- The penalty grows with the count of each token so far.
- It is subtracted from the token’s logit, unlike the multiplicative repetition penalty.
- Typical useful values are 0.1 to 0.8.
- Negative values encourage repetition, which is occasionally useful.
- It pairs with presence penalty, which triggers once per token regardless of count.
Frequency penalty explained in simple terms
Think of a writing coach who gently discourages overused words. Using “innovative” once is fine; by the fifth use, they push back harder. The pressure scales with how often you have reached for the same word.
How it works
At each generation step, for every token that has already appeared, the model’s logit is reduced by the penalty value multiplied by the number of times that token was used. Softmax then converts the adjusted logits to probabilities.
Worked example with a frequency penalty of 0.5:
| Token | Times used so far | Logit reduction |
|---|---|---|
| ” solution” | 1 | 0.5 |
| ” platform” | 3 | 1.5 |
| ” scalable” | 6 | 3.0 |
“Scalable” is heavily suppressed, “solution” only slightly. An unused token receives no reduction at all.
Choosing a value
| Value | Effect |
|---|---|
| 0 | No penalty |
| 0.1–0.3 | Light variety, safe for most writing |
| 0.4–0.8 | Noticeably more varied vocabulary |
| 1.0+ | Can distort grammar and force unnatural word choices |
| Negative | Encourages repetition, useful for chants, formats or lists |
Frequency vs presence vs repetition penalty
| Setting | Trigger | Scaling | Typical source |
|---|---|---|---|
| Frequency penalty | Token already used | Grows with count | Commercial APIs |
| Presence penalty | Token used at least once | Fixed, one-time | Commercial APIs |
| Repetition penalty | Token already used | Multiplicative on logits | Open-source tools |
Use frequency penalty to reduce overuse of specific words, and presence penalty to push the model towards new topics.
When to use it
- Long-form content where vocabulary becomes repetitive.
- Marketing copy that overuses buzzwords.
- Generating lists of varied ideas.
- Chat assistants that repeat stock phrases.
When to avoid it
- Code generation, where identifiers must repeat.
- Structured output with repeated keys.
- Technical writing where the correct term must be reused.
- Translation, where faithfulness matters more than variety.
Limitations
- Cannot distinguish desirable from undesirable repetition.
- Penalises common function words along with buzzwords.
- Effects compound in long outputs.
- Support and exact behaviour differ between providers.