Track 08 · Deployment & opsLesson 02 of 11
Containers and Kubernetes for agents
The runtime for anything that outlives a request, set up for production.
- Lesson library · reference
- 30 min read
- 75 min hands-on
- Pro
What you'll be able to do
01- You can build an agent image a platform team will admit: slim base, non-root, pinned base digest, deployed by digest.
- You can choose the Kubernetes primitive for each part of an agent system and explain why.
- You can design liveness, readiness and startup probes for a model-dependent workload, and defend keeping the model out of them.
- You can implement graceful shutdown that drains in-flight runs within the grace period and checkpoints the rest.
- You can configure queue-depth autoscaling with a ceiling derived from model quota.
- You can choose between EKS, ECS on Fargate, Cloud Run and Lambda, citing the limits that decide it.
Before you start
- Workflows that survive restarts: checkpointing and replay, applied here at the pod boundary.
- Retries, fallbacks and failing gracefully: idempotency keys and circuit breakers.
- Secrets, dependencies and network isolation: signing, SBOMs, default-deny egress.
- Deploying into the customer's cloud: the accounts and network the cluster lives in.
- Background: you have built a Docker image and run
kubectl apply.
The short version
An agent run is not a web request. It can take minutes, wait on a slow model, drive a browser and pause for a human. Kubernetes treats pods as disposable: it drains nodes, rolls deployments and kills containers that exceed memory, with 30 seconds' notice by default. Production agent runtime reconciles those facts. Ship a small non-root image by digest. Pick workload types by run length. Keep the model out of health checks. Scale on queue depth, capped at what model quota can serve. Treat SIGTERM as routine: stop taking work, finish what fits, checkpoint the rest, let another pod resume.
- A pod is a lease, not a home. Runs longer than the grace period must survive the pod ending.
- Liveness checks the process, readiness checks this pod, neither checks the model.
- Agents are I/O-bound. CPU autoscaling barely moves; queue depth is the signal.
- Graceful shutdown is a budget. Size it to the longest step, not the longest run.
- The runtime is the customer's. Their blueprint, admission policies and node upgrades set the rules.
Pro lesson
The rest of this lesson is in Pro.
The mental model, worked example, field note, failure table, exercise and self-check, with the voice tutor.
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI