Supervised fine-tuning (SFT) is the process of further training a pretrained model on labelled examples, usually prompts paired with ideal responses, so it learns a desired behaviour, format or task. In LLM development, SFT is typically the first step after pretraining, and it is also the most common way companies customise models on their own data.
Key takeaways
- SFT uses supervised learning: each input has a target output.
- For LLMs, data is usually prompt-response or multi-turn chat examples.
- Instruction tuning is a type of SFT focused on following instructions broadly.
- SFT usually comes before preference tuning such as RLHF or DPO.
- It can be done as full fine-tuning or with parameter-efficient methods such as LoRA.
Supervised fine-tuning explained in simple terms
Pretraining is like reading a whole library. SFT is like apprenticeship with a set of worked examples: “here is the request; here is exactly how an expert would respond.” The model studies these examples until it reliably produces similar responses to similar requests.
Where SFT fits in LLM training
| Stage | Data | Purpose |
|---|---|---|
| Pretraining | Trillions of unlabelled tokens | Broad knowledge |
| Supervised fine-tuning | Thousands to millions of prompt-response examples | Follow instructions; learn formats and tasks |
| Preference tuning (RLHF, DPO) | Preferred vs rejected responses | Improve helpfulness, tone and safety |
| Optional RL for reasoning | Tasks with checkable answers | Stronger step-by-step reasoning |
How SFT works
- Collect examples of high-quality prompt-response pairs for the target behaviour.
- Format them consistently, often in the model’s chat template with system, user and assistant roles.
- Train: the model predicts the response tokens given the prompt. The loss is usually computed only on the response, so the model learns to answer rather than to repeat prompts.
- Validate on held-out examples and real-world test prompts.
- Iterate: fix gaps by adding targeted examples.
A worked example: support reply style
A SaaS company wants replies that are short, friendly and always end with a relevant help link. They prepare 2,000 examples:
Prompt: “Customer: I can’t export my contacts to CSV.”
Response: “Sorry for the trouble! Go to Contacts → Export and choose CSV. If the button is greyed out, check that your role has export permission. Guide: [link]”
After SFT, the model follows this pattern for new questions without long prompts. The company still grounds facts with retrieval, because SFT teaches style and format better than it teaches changing facts.
What makes good SFT data
- Quality over quantity: a few thousand excellent examples can beat many mediocre ones.
- Coverage: include edge cases, refusals and “I don’t know” answers.
- Consistency: same format and tone throughout.
- Correctness: errors in examples become errors in the model.
- Realism: use prompts that look like real user inputs.
SFT vs other approaches
- vs prompt engineering: prompting is faster and cheaper to change; SFT gives more consistent behaviour and shorter prompts at scale.
- vs retrieval (RAG): RAG supplies current facts; SFT teaches behaviour and format.
- vs preference tuning: SFT shows correct answers; preference tuning teaches which of several answers is better.
Limitations
- Needs well-curated labelled examples.
- Poor at teaching large amounts of new or changing knowledge.
- Can cause catastrophic forgetting, weakening abilities not covered by the data.
- Can make a model overconfident if examples never show uncertainty.