Track 02 · RetrievalLesson 07 of 11
Retrieval that respects who can see what
Filter by the user's document permissions before anything reaches the model.
- The FDE path · Stage 3 of 6
- 35 min read
- 75 min hands-on
- Pro
What you'll be able to do
01- Explain why retrieval must reproduce the source system's permissions exactly, and name the paths by which restricted content leaks (answers, citations, snippets, caches, logs, shared channels).
- Design ACL capture at ingest for SharePoint/OneDrive, Google Drive and Confluence, including inheritance, nested groups and identity mapping.
- Choose between early-binding and late-binding enforcement, and between pre-filtering and post-filtering, and defend the recall and leakage trade-offs.
- Implement a deny-by-default, ACL-filtered retriever that propagates the end user's identity, expands nested groups and scopes caches by permission set.
- Build a leakage test suite with canary documents and red-team queries that runs in CI.
- Quantify revocation latency and agree it with a customer's CISO as a measurable target.
Before you start
- Parsing and chunking documents that fight back: ACLs are metadata you capture at ingest, alongside content.
- Vector stores and HNSW tuning: payload/metadata filtering and its effect on ANN recall.
- Retrieval that plans, checks and retries: every retrieval tool the planner calls must enforce the rules in this file.
- Background: OAuth 2.0 / OIDC tokens at the level of "the app receives a signed token that says who the user is".
Org-wide authorisation models (RBAC, ABAC) and tenancy isolation are owned by Authorization and multi-tenancy. This file owns the retrieval layer: how document permissions get into the index and how they are enforced on every query.
The short version
A retrieval system that can read everything must never answer as if the user can read everything. Capture each document's ACL from the source at ingest (users, groups, deny entries, inheritance resolved), store it as metadata on every chunk, and at query time filter on the principals of the authenticated end user, expanded through nested groups. Fail closed: no ACL, unknown principal or unverified identity means no access. Filter inside or before the search, never only after it, so unauthorised text never reaches a reranker, a log, a cache or the model. Measure revocation latency, scope every cache to the permission set, and prove it all with canary documents and red-team queries in CI.
Key takeaways:
- The acceptance criterion is parity with the source: retrieval-visible must be a subset of source-visible for every user, every time.
- Use the user's identity, not the agent's. The ingestion service account can read everything; that is exactly why it must not be used at query time.
- Store group IDs on documents and expand the user's groups at query time. Flattening groups into per-document user lists turns every membership change into a mass re-index.
- Early binding (filter on indexed ACLs) is fast but stale by the sync interval; late binding (check with the source at result time) is current but slow. High-sensitivity sources often need both.
- Semantic caches and shared chat channels are the leaks most teams miss.
Pro lesson
The rest of this lesson is in Pro.
The mental model, worked example, field note, failure table, exercise and self-check, with the voice tutor.
Also in Pro
- Every stage and specialisation, in full
- Every lab and every role project
- Halden weeks 2–10, on your record
- Every drill, live in the playground
- All solutions, plus the server judge
- Lab Assistant and every Teach with AI