Track 08 · Deployment & opsLesson 10 of 11
SLOs and latency budgets
Agree the reliability numbers before go-live, not after the first complaint.
- The FDE path · Stage 6 of 6
- 45 min read
- 75 min hands-on
- Pro
What you'll be able to do
01- Explain SLI, SLO, SLA, error budget and burn rate, and where the internal objective ends and the contractual agreement begins.
- Choose SLIs for an agent workflow (task availability, per-call and per-task latency, time to first token where a person is waiting, task quality, freshness) and write each one as good events over valid events.
- Implement error-budget accounting and multi-window, multi-burn-rate alert evaluation, and calculate how long an alert takes to fire for a given error rate.
- Design a latency budget across the agent loop and explain why sequential calls amplify tail latency.
- Defend a reliability target in front of the customer by composing dependency availability, and write the error budget policy they sign.
Before you start
- Tracing agents with OpenTelemetry: SLIs are computed from the spans and metrics you emit there.
- The agent loop, written by hand and Retries, fallbacks and failing gracefully: step budgets, timeouts, retries and degradation paths. This concept turns them into agreed numbers.
- A gateway for bursty agent traffic: the limits that protect upstream systems and shape your dependency targets.
- From requirements to eval criteria and Catching drift before the customer does: where quality signals come from.
- Comfort with percentiles and basic probability (independent events, products of probabilities).
The short version
An SLO is a number you and the customer agree before go-live: what fraction of real work must be good, measured how, over what window. The SLI is the measurement, the SLO is the target, and the error budget is the failure you may spend on incidents, changes and dependency outages. Burn-rate alerts page when the budget is going fast enough to matter; the error budget policy says what changes when it runs low. For agents, "good" includes latency and outcome, not just HTTP 200, and the latency budget is split across sequential model calls, tools and guardrails whose tails compound. Targets must be achievable on top of dependencies you do not control.
- Write SLIs as good/valid events at a measurement point the user recognises, never as averages.
- Turn percentile targets into proportions ("95% of model calls within 8 s") so they have a budget and can be alerted on.
- Page on burn rate over two windows, not on raw error rate. Ticket on slow burns.
- If each of six sequential calls has a 5% chance of being slow, about a quarter of runs see at least one slow call.
- The SLO is internal and stricter; the SLA is the contract, owned by Handover, run state and the next use case.
Pro lesson
The rest of this lesson is in Pro.
The mental model, worked example, field note, failure table, exercise and self-check, with the voice tutor.
Pro opens this stage
Stage 6 · Ship, run and hand over- 8 lessons, with the voice tutor
- Lab · Check a trace export before you trust the dashboard, graded here
- Halden week 7 · The champion moves
- Halden week 8 · The portal goes down before cut-off
- Halden week 9 · Prices up, and a new rule
- Halden week 10 · A second site, and handover
- The lab kit, to run Halden weeks on your machine
Also in Pro
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI