Fine-tuning is not always the right tool. Before committing to it, exhaust prompt engineering. Fine-tune when you have a specific, consistent behavior requirement that prompts alone cannot reliably deliver.
A well-crafted system prompt with few-shot examples is cheaper, faster, and easier to iterate than fine-tuning. Only move to fine-tuning when you've hit the limit of what prompting can achieve.
Consistent output format — Downstream code parses model output and breaks on format deviations. A fine-tuned model trained on thousands of correct-format examples deviates far less than a prompted base model.
Tone and brand voice — Rather than including lengthy tone instructions in every prompt (adding cost, imperfectly followed), a fine-tuned model internalizes the voice and produces it naturally.
Model distillation — Use a large frontier model to generate high-quality training data, then fine-tune a smaller, cheaper, faster model to replicate that behavior on your specific task.
Specialized reasoning — Clinical diagnostic protocols, legal argument structures, and domain-specific reasoning frameworks that prompts alone cannot teach reliably.
System prompt cost — If achieving the behavior requires a 3,000-token system prompt on every request, fine-tuning that behavior into the model reduces per-request cost significantly at scale.
When the need is about knowledge — current information, private documents, facts that change. Fine-tuning is a poor and expensive mechanism for injecting knowledge.
When the behavior is achievable with a well-crafted system prompt. When you lack a high-quality labeled dataset. When you need fast iteration.
Ask the AI assistant about when to fine-tune, model distillation, or the tradeoffs between prompting and fine-tuning.