Track 01 · Models & contextLesson 02 of 10
Choosing a model you can defend
Gates, evidence and an exit plan, so “why this model?” has a five-part answer.
- Lesson library · reference
- 35 min read
- 60 min hands-on
- Pro
What you'll be able to do
01- Explain the practical differences between frontier closed models, open-weight models and managed hosting of either, in terms of data handling, control, cost and operational burden.
- Describe any model by tier and capability dimensions instead of by leaderboard position.
- Choose a hosting route (direct provider API, cloud marketplace, self-hosted open weights) that satisfies a customer's residency, procurement and security constraints.
- Design and run a selection process: hard gates, a task-specific mini eval with repeated runs, and a weighted matrix with sensitivity analysis.
- Defend "why this model?" in an architecture review, including the version-pinning, fallback and deprecation plan.
Before you start
- Tokens, context windows and why answers go wrong: tokens, pricing classes, latency anatomy, non-determinism.
- Familiarity with at least one hosted LLM API.
The short version
There is no best model, only a best-fit model for a task, a constraint set and a date. Start from the customer's hard constraints (where data may be processed, what licence terms legal will accept, which languages must work, what security review requires) and eliminate anything that fails them. Shortlist three to five candidates across tiers and hosting routes. Run a small eval built from the customer's own tasks, several times per item, measuring quality, p95 latency and cost per task. Score against the requirements with weights agreed before you see results, and check whether the decision survives a change in weights. Then write down the pinned version, the fallback, the owner and the date you will re-run the eval, because the model you choose will be deprecated.
- Gates before weights. Residency, licence, languages and must-pass safety items are pass/fail. No score can buy them back.
- Benchmarks shortlist; your eval decides. Public numbers rarely measure the customer's task.
- Describe tiers and capabilities, not ranks. Rankings expire within months.
- Hosting route is a first-class decision. The same model can be acceptable via one route and blocked via another.
- Every choice ships with an exit plan: pinned ID, fallback, re-evaluation date.
Pro lesson
The rest of this lesson is in Pro.
The mental model, worked example, field note, failure table, exercise and self-check, with the voice tutor.
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI