Human review and feedback loops

Automated evals scale; humans define what "good" means. A production AI system needs a standing review programme, not occasional spot checks: named reviewers, a cadence, a sampling plan, written guidelines, measured agreement, queues with SLAs, and a route from every finding to a fix. Sample in slices. A uniform random slice gives you an unbiased estimate of quality. Targeted slices (high-risk intents, low judge scores, judge-versus-user disagreements, policy flags) find problems faster but overstate the error rate if you read them as a measurement. Write guidelines with decision rules and boundary examples, calibrate reviewers on shared items, and measure inter-annotator agreement with kappa or Krippendorff's alpha. If trained reviewers cannot agree, the metric is not real yet. Subject-matter expert time is the scarcest resource on the engagement, so spend it on adjudication and hard cases, with context pre-assembled. Redact what reviewers do not need, keep reviewing in the approved region, and watch for fatigue and for reviewers who simply agree with the machine.

• The random slice measures; the targeted slices find. Keep both, and never mix them in one number.
• Agreement before automation. A label set with low inter-annotator agreement cannot calibrate a judge or gate a release.
• Guidelines are versioned artefacts. Every label records the guideline version it was made under.
• Every finding has a destination. A review that does not change a dataset, a prompt, a corpus or a policy was an expense, not a feedback loop.
• Reviewers are users of production data. Least privilege, redaction, residency and audit logs apply to them.