Track 01 · Models & contextLesson 03 of 10
Reasoning models and what thinking costs
Set reasoning effort per route from eval evidence, not from the default.
- Lesson library · reference
- 35 min read
- 45 min hands-on
- Pro
What you'll be able to do
01- Explain what reasoning models do differently from standard models, and why thinking tokens cost money and latency even when you never see them.
- Configure reasoning depth on Anthropic and OpenAI using the current parameters, and read the usage fields that show what it cost.
- Choose where reasoning earns its cost and where it does not, and prove it with an eval rather than intuition.
- Design a difficulty router that keeps the easy majority of traffic on cheap, fast settings and escalates the rest.
- Defend a reasoning budget to a CFO with a cost-per-correct-answer comparison.
Before you start
- Tokens, context windows and why answers go wrong: output pricing, latency anatomy, stop reasons.
- Choosing a model you can defend: tiers and task-specific evals.
The short version
Reasoning models generate internal "thinking" tokens before, and often between, the tokens you see. On hard multi-step problems this raises accuracy; on easy ones it mostly raises the bill and the wait. Thinking is billed as output, the most expensive class, and delays the first visible token. Providers expose depth controls (Anthropic's effort with adaptive thinking; OpenAI's reasoning.effort), and some current models cannot switch thinking off. So treat depth as a per-route setting: measure accuracy by difficulty and effort, send the easy majority to low effort, reserve high effort for slices where the eval shows a gain, and cap total output.
- Thinking tokens are output tokens. Billed at output rates, whether or not they are returned to you.
- Depth is a per-route decision, backed by an eval by difficulty and effort.
- Most enterprise traffic is easy. Paying for deep reasoning on it buys latency, not accuracy.
- Do not prompt a reasoning model to "think step by step". It already does; tell it the goal and constraints.
- Summarised reasoning is not an audit trail. Evidence and deterministic checks are.
Pro lesson
The rest of this lesson is in Pro.
The mental model, worked example, field note, failure table, exercise and self-check, with the voice tutor.
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI