Keeping the knowledge base fresh

A knowledge base is a copy of the customer's documents, and copies go stale. When a policy changes and the index does not, the assistant keeps answering fluently with the old rule; nothing errors, nothing alerts, and the first signal is a complaint weeks later. Freshness is an engineering system: detect changes per source (webhooks, delta APIs, CDC, or polling with ETags and checksums), plan the minimum work (content changes re-embed; permission changes update metadata; unchanged documents are skipped), handle deletions with tombstones (including right-to-erasure requests), keep versions and effective dates, and re-index at scale behind an alias so you can switch and roll back without downtime. Then measure it: last successful sync and index lag per source, against SLAs the customer agreed.

Key takeaways:

• Every source needs an explicit freshness SLA, and every chunk needs source_modified_at and indexed_at so you can measure against it.
• Content hash, ACL hash and pipeline version are three separate change signals with three different costs.
• Absence from a listing is not proof of deletion. Only a complete listing or an explicit delete event justifies removing content, and a mass-delete guard stops a broken connector from emptying the index.
• Embedding-model migrations are releases: new index, dual writes, backfill, offline evaluation, alias swap, keep the old index for rollback.
• Deletion must reach every copy: vectors, keyword index, caches, graph facts and backups within their retention window.