Prompt tuning is a parameter-efficient fine-tuning method that learns a small set of continuous vectors, called a soft prompt, which are added in front of the input while the model’s weights stay frozen. These vectors are not words. They are trainable numbers in the same space as token embeddings, optimised by gradient descent to steer the model towards a task.
Key takeaways
- Only the soft prompt is trained, often a few thousand to a few hundred thousand parameters.
- Soft prompts are continuous vectors, not readable text.
- It was introduced by Brian Lester and colleagues at Google in 2021.
- Effectiveness improves sharply with model size.
- “Soft prompt tuning” is another name for the same method.
Prompt tuning explained in simple terms
Prompt engineering is like writing better instructions in English. Prompt tuning is like handing the model a small set of coded signals it understands better than any English phrase, discovered automatically through training. You never read them, but they reliably put the model in the right frame of mind for your task.
How it works
- Choose a soft prompt length, commonly 10 to 100 virtual tokens.
- Initialise their vectors, sometimes from real word embeddings.
- Prepend them to every input’s embedded tokens.
- Train only these vectors on task examples; the base model never changes.
- Save the soft prompt, typically a file of a few megabytes or less.
Worked example: with an embedding dimension of 4,096 and 20 virtual tokens, the soft prompt has 20 × 4,096 = 81,920 trainable parameters. That is roughly 0.001% of a 7-billion-parameter model.
Prompt tuning vs related methods
| Method | What is trained | Where it applies |
|---|---|---|
| Prompt engineering | Nothing; text only | At the input, in words |
| Prompt tuning | Soft prompt vectors | At the input embeddings |
| Prefix tuning | Vectors added at every layer | Inside each attention layer |
| LoRA | Low-rank weight updates | Inside existing weight matrices |
| Full fine-tuning | All weights | Everywhere |
When it works well
The original paper found that prompt tuning nearly matched full fine-tuning once models reached billions of parameters, while remaining far cheaper. Smaller models benefited much less. It suits settings where you want many task-specific variants of one frozen model, since each variant is only a small vector file.
Practical notes
- Soft prompts consume part of the context window.
- They are tied to the exact base model that trained them.
- Training is sensitive to the learning rate, usually higher than for full fine-tuning.
- Results are harder to inspect than text prompts, since the vectors are not readable.
Limitations
- Generally weaker than LoRA on complex tasks, which is why LoRA dominates today.
- Needs large base models to work well.
- Cannot be understood or edited by hand.
- Adds tokens to every request, using context and compute.