Three tools for three different problems, not three tiers of the same one
These three get compared as if they were ranked options, prompting for beginners, RAG for intermediate, fine-tuning for serious production use. They're not a ladder. They solve different problems, and the honest first question is which problem you actually have.
Changing behavior, format, or tone, and doing it fast, with the ability to change your mind tomorrow, is prompting's job, covered in Prompting as Interface Design earlier in this series. Grounding answers in facts that live in a specific corpus and change over time is RAG's job, covered two articles ago, along with an honest account of when it's the wrong tool. Fine-tuning is neither of those. It earns its place only when the model's default behavior itself needs to shift in a way prompting can't reach reliably, or when cost and latency justify replacing a long prompt on a large model with a short one on a smaller, specialized one.
Most teams that reach for fine-tuning first haven't actually exhausted the first two. The rest of this article is about what fine-tuning is actually for, so that reach is a decision, not a reflex.
Fine-tuning teaches a pattern of behavior, not a fact
The most common misconception about fine-tuning is treating it like a way to teach a model new facts, upload your company's documentation and the model will "know" it. That's RAG's job, and fine-tuning is a genuinely bad tool for it, a model fine-tuned on a set of facts doesn't reliably recall them any better than an unmodified model with those same facts retrieved and placed in context.
What fine-tuning changes is behavior: shifting the model's default response pattern given hundreds or thousands of examples of input paired with the exact output you want.
One example from a fine-tuning dataset, teaching a specific output format
1{"messages": [2 {"role": "user", "content": "Customer says the app crashes on login."},3 {"role": "assistant", "content": "{\"category\": \"bug\", \"severity\": \"high\", \"component\": \"auth\"}"}4]}
A thousand examples like this teach the model to default to this exact triage format for this exact kind of input, reliably, without a lengthy system prompt spelling out the schema on every call. That's a real, valuable shift. It's a shift in behavior pattern, not an injection of new facts, and treating it as the latter is how a team ends up disappointed that their fine-tuned model still doesn't know something that was never going to stick this way in the first place.
What most people mean by "we fine-tuned a model"
Full fine-tuning, updating every weight in the model, is expensive enough in compute and infrastructure that it's mostly done by the labs training foundation models themselves, not by a team building a product on top of one.
LoRA, low-rank adaptation, is what almost everyone actually means when they say they fine- tuned a model. The base model stays frozen, untouched, and a small set of additional parameters gets trained on top of it, dramatically cheaper to train and to store, since you're saving a small adapter file rather than a full copy of the model's weights.
The practical workflow is a few hundred to a few thousand high-quality examples of the exact behavior you want, more isn't automatically better if the examples are inconsistent with each other, a training run against those examples, and then the same golden-set evaluation from earlier in this series, run against the fine-tuned model the same way it would run against a prompt change. A fine-tuned model is still a model producing output that needs checking, not a graduation past needing evals.
The honest cost comparison almost always favors prompting first
A prompt change ships in minutes and reverts in minutes. Fine-tuning requires a real dataset, which means someone building it carefully, a training run, a full pass through the golden set, and a plan for retraining every time requirements shift, because a fine-tuned model doesn't absorb a new instruction the way a prompt does, it needs new examples and a new run.
That cost is worth paying when prompting has genuinely been tried and genuinely can't hold the behavior reliably enough, a format that needs to be exactly right on every single call at a volume where the occasional prompt-following slip is unacceptable, or a cost and latency target that only a smaller, specialized model can hit. It's not worth paying because a prompt felt fragile after one round of iteration, or because fine- tuning sounds like the more serious, production-grade choice.
The bar worth holding: prompting and RAG have both been genuinely exhausted for this specific problem, and there's a real dataset and a real appetite to maintain a fine-tuned model going forward, not just for the one time it gets trained.