A probability distribution in an LLM is the set of probabilities the model assigns to every possible next token at a given step, with all values adding up to 1. It is produced by applying softmax to the model’s logits. Everything about how a model generates text, including creativity, determinism and reliability, comes down to this distribution and how a token is selected from it.
Key takeaways
- Every generation step produces one probability per vocabulary token.
- The distribution can be sharp, with one dominant token, or flat, with many plausible options.
- Temperature, top-k and top-p reshape or trim the distribution before sampling.
- Entropy measures how uncertain the model is at that step.
- Calibration is how well stated probabilities match real-world correctness.
The distribution explained in simple terms
Imagine a weather forecast that gives probabilities for every possible outcome: sunny 70%, cloudy 20%, rain 9%, snow 1%. An LLM does this for words. After “The capital of France is”, the distribution might be sharply peaked on ” Paris”. After “My favourite colour is”, it is spread across many colours, because many continuations are reasonable.
Sharp vs flat distributions
| Situation | Distribution shape | Meaning |
|---|---|---|
| Factual completion with one right answer | Sharp: one token near 1 | The model is confident |
| Open-ended creative writing | Flat: probability spread widely | Many valid continuations |
| Ambiguous or unfamiliar context | Moderately flat | Possible uncertainty or a knowledge gap |
Entropy puts a number on this spread. Low entropy means a confident, peaked distribution; high entropy means many plausible options.
A worked example
After “Please find the attached”, a model’s top probabilities might be: ” invoice” 0.31, ” document” 0.22, ” file” 0.18, ” report” 0.11, with the rest spread thinly across thousands of tokens. Greedy decoding always picks ” invoice”. Sampling might pick ” file” sometimes, which is why the same prompt can produce different wording each time.
How sampling settings reshape the distribution
| Setting | Effect on the distribution |
|---|---|
| Temperature below 1 | Sharpens it, favouring the top tokens |
| Temperature above 1 | Flattens it, giving rare tokens more chance |
| Top-k | Keeps only the k highest-probability tokens, then renormalises |
| Top-p (nucleus) | Keeps the smallest group whose probabilities reach p, then renormalises |
| Repetition and frequency penalties | Reduce probabilities of tokens already used |
| Logit bias | Raises or lowers specific tokens before softmax |
Calibration
A model is well calibrated if things it says with 80% probability turn out true about 80% of the time. Research has found that base models are often reasonably calibrated on next-token prediction, while heavy preference tuning can make models overconfident. This matters when using token probabilities as confidence scores for routing or review, so calibration should be checked on your own data.
Practical uses
- Flag low-confidence answers for human review.
- Choose between candidate completions in classification tasks.
- Detect uncertain extractions in document processing.
- Tune sampling settings for factual versus creative tasks.
Limitations
- Confidence in the next token is not confidence in the whole answer.
- Models can be confidently wrong, which is one root of hallucination.
- Distributions differ between models, so thresholds do not transfer.
- Most APIs expose only the top few probabilities, not the full distribution.