Track 09 · Systems & unit costLesson 08 of 11
When to fine-tune, and when not to
Data, technique and serving, once the cheaper options are ruled out.
- Specialisation · Data & AI platform
- 40 min read
- 75 min hands-on
- Pro
What you'll be able to do
01- Choose between prompting, retrieval, tooling, routing and fine-tuning for a given requirement, using a decision ladder you can defend to a CFO and a model-risk reviewer.
- Explain what fine-tuning reliably changes (format, style, label consistency, prompt length, model size) and what it does not (fresh or permissioned knowledge).
- Implement a dataset-prep pipeline from production traces: consent filter, PII scrub, dedup, length filters, conversation-grouped stratified splits and a leakage check, exported as chat-format JSONL.
- Compare full SFT, LoRA/QLoRA, DPO and reinforcement fine-tuning, and state which providers offer which as of September 2026.
- Design the evaluation and serving plan for a tuned model: three-way comparison against the base and the prompt-only baseline, multi-LoRA or provider hosting, and a re-tune plan for when the base model is retired.
Before you start
- Golden datasets and the metrics that matter and Regression tests and CI gates: the tuned model is judged on the same held-out set as everything else.
- Finding and masking personal data: training data is the highest-risk place for PII to end up.
- Treat prompts like releases: the rung below fine-tuning on the ladder.
- Cost per resolved case and Making it feel fast: the cost and latency case for distillation.
- Background: you know what a loss function and a train/validation split are. You do not need to have trained a transformer.
The short version
Fine-tuning changes a model's habits, not its knowledge. It is the right tool when a well-prompted model already knows enough but is inconsistent in format, style or labels, or when a smaller tuned model can match a large prompted one at a fraction of the cost and latency. It is the wrong tool for facts that change, facts that need citations, or facts that only some users may see: those belong in retrieval. Climb the ladder in order (evals, prompt, retrieval, tools, routing, then tuning), and when you do tune, treat the dataset, pipeline and eval as the asset, because the weights expire when the base model is retired.
- Build the eval set and measure a prompt-only baseline before anyone says "fine-tune".
- Good reasons: format and schema adherence, fixed taxonomies, tone, shorter prompts, distilling a large model into a small one.
- Bad reasons: "teach it our policies", "make it know our products", "stop hallucinations".
- Dataset prep is most of the work: consent, PII scrub, dedup, label quality, and splits grouped by conversation so the test set is not leaked.
- Every tuned model has a re-tune date: the day its base model is deprecated.
Pro lesson
The rest of this lesson is in Pro.
The mental model, worked example, field note, failure table, exercise and self-check, with the voice tutor.
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI