SFT trains the model on a dataset of input-output pairs, where each example shows exactly what the model should produce given a specific input. The model learns the desired output patterns by minimizing prediction error across thousands of examples.
SFT data is structured as prompt-completion pairs — typically formatted as conversation turns:
System: "You are a concise customer support agent for Acme Software."
User: "My account is locked."
Assistant: "Go to the login page, click 'Forgot Password', and check your email within 5 minutes. Contact us if you don't receive it."
The model is trained to predict the assistant turn. Loss is computed only on the response tokens — not the input — so the model learns to generate, not memorize prompts.
500 carefully curated, diverse, high-quality examples typically outperform 10,000 noisy or redundant ones. The most common SFT mistake is using too much low-quality data rather than too little high-quality data.
Hundreds to a few thousand examples. Getting the model to consistently use a specific JSON structure or tone requires fewer examples than complex task adaptation.
Tens of thousands of examples. Teaching structured clinical reasoning, legal analysis, or multi-step problem solving requires broader coverage of the task space.
Human-written — Highest quality. Domain experts write ideal responses. Most expensive and time-consuming.
Frontier model distillation — Use GPT-4 or Claude to generate responses, humans review and filter, use approved pairs for SFT of a smaller model. Scalable and effective.
Production logs — Quality-filter real interactions from a deployed model or human agents. Real distribution of user queries, but requires careful curation.
Ask the AI assistant about supervised fine-tuning, data formatting, dataset size, or generating training data through distillation.