Interview drill 11Interview-style design round
A support agent got worse after a model upgrade: find out why by Friday
- 45 minutes
- Every elective
- No real companies
Hypothetical. No affiliation with or endorsement by the companies named. Public facts as of 6 October 2026; sources at the end of the brief.
The prompt
"You're the forward deployed engineer who helped deploy a customer-support agent at a US subscription software business. Last Tuesday the platform team moved the agent to a newer version of the same model. This week's business review shows the resolution rate, the share of conversations closed without a human, fell from 64% to 56%. The VP of support wants a decision by Friday: roll back or not, and why it happened. Rolling back isn't free, because the provider retires the old version in six weeks. You have traces from an OpenTelemetry pipeline, an offline eval suite that passed before the upgrade, and the ticketing data. It's Tuesday morning. Walk me through what you'd do between now and Friday."
The clock
- Clarify0–5 min
- Discovery plan5–15 min
- Design15–30 min
- Risks and evals30–40 min
- Rollout and the ask40–45 min
Lessons for this drill
Pro drill
The rest of this drill is in Pro.
What to know going in, the curveballs on the clock, a strong answer, the common mistakes and the rubric.
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI