Many production systems that use fine-tuning would perform just as well with a good system prompt and RAG — at a fraction of the engineering investment. Knowing when not to fine-tune is as important as knowing when to do it.
You haven't tried prompting first — A detailed system prompt with 3–5 few-shot examples is hours of work. Fine-tuning is days to weeks. Exhaust prompting before committing to fine-tuning.
The problem is knowledge, not behavior — If the model gives wrong answers because it lacks information, fine-tuning won't fix it reliably. Use RAG. Fine-tuning is for behavior, not knowledge injection.
The dataset is too small — Fewer than a few hundred high-quality, diverse examples for a non-trivial task means severe overfitting. A prompt with 10 good few-shot examples often outperforms fine-tuning on 50 poor ones.
Requirements change frequently — Updating a fine-tuned model requires retraining, re-evaluation, and redeployment. A prompt can be updated in minutes. Build a prompt management system instead.
A newer base model solves the problem — Always test on the current best base models before concluding fine-tuning is needed. Many tasks fine-tuning solved on older models are handled out-of-the-box by newer ones.
Cost-benefit doesn't justify it — Calculate the metric improvement from fine-tuning vs a strong prompt. If the margin is small, the ongoing maintenance burden of a fine-tuning pipeline may not be worth it.
Have you tried a detailed system prompt with few-shot examples? Is the problem behavior (not knowledge)? Do you have hundreds of high-quality, diverse examples? Are requirements stable enough for a training pipeline? Would a newer model solve it without fine-tuning? If the answer to any of these is no, don't fine-tune yet.
Ask the AI assistant about when to use prompting vs fine-tuning vs RAG, or how to evaluate whether fine-tuning is justified for your use case.