Track 04 · Multi-agent & integrationLesson 07 of 11
A gateway for bursty agent traffic
Quotas, rate limits and fallbacks that protect the customer's upstream systems.
- Lesson library · reference
- 40 min read
- 75 min hands-on
- Pro
What you'll be able to do
01- Explain why agent traffic arrives in correlated bursts, and estimate the peak a cut-off fan-out puts on each upstream.
- Choose a rate-limiting algorithm for a given limit, and show how burst size changes what "5 per second" means.
- Design per-tenant, per-agent and per-tool quotas, concurrency limits and bulkheads, with priority queueing, backpressure and load shedding.
- Implement an offline gateway simulation that proves the customer's limits hold under a burst and reports who was accepted, queued, cached or rejected.
- Defend the design to a system owner: an operating agreement, fallbacks that lower autonomy, a kill switch they can pull, and metrics they can read.
Before you start
- Retries, fallbacks and failing gracefully: retries with jitter,
Retry-After, retry budgets, idempotency and circuit breakers inside one agent. Assumed here, not re-taught. - MCP: connecting agents to the customer's systems: Streamable HTTP and the 2026-07-28 revision, whose routing headers the gateway uses.
- Tools with scoped, auditable access: one identity per agent; quotas key on it.
- SaaS connectors and auth flows and Automating systems with no API: what the gateway sits in front of.
- Agents started by events and schedules: where bursts come from, and where backpressure goes.
The short version
The customer's systems were sized for people working one case at a time. Agents turn one business event into hundreds of near-simultaneous calls on SAP, ServiceNow and a carrier portal, and retries, loops and partner agents add more. An agent gateway is the shared control point between all your agents and those systems. It identifies the caller, applies per-tenant, per-agent and per-tool quotas, limits both the rate of new calls and the number in flight, queues briefly within the caller's deadline, sheds low-priority work first, serves safe reads from cache, switches the fleet to a fallback mode when an upstream degrades, enforces cost caps and kill switches, and emits the metrics that prove the limits held.
Key takeaways:
- Size limits from the upstream's capacity and its owner's agreement, never from your agent count. Write each as rate, burst, scope and source.
- You need a rate limit and a concurrency limit; a slow upstream defeats a rate limit on its own.
- Queue only within the caller's deadline; otherwise reject fast with a reason and
Retry-After. - Tenant and priority come from authenticated identity and the deterministic router, never from the model.
- The target number of throttles you receive from an upstream is zero.
Pro lesson
The rest of this lesson is in Pro.
The mental model, worked example, field note, failure table, exercise and self-check, with the voice tutor.
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI