Retrieval that plans, checks and retries

Classic RAG runs one retrieval with the user's words and hands whatever comes back to the model. Agentic RAG treats retrieval as a set of tools the model chooses to call: it decides whether retrieval is needed, plans what to look for, picks a source, grades what came back, and retries with a better query or a different source when the evidence is weak. It stops when the evidence is sufficient, when a budget runs out, or when it stops making progress, and then answers with citations or abstains with the gap stated. The engineering work is not the loop itself. It is the grader, the sufficiency definition, the stopping policy and the budget, because those decide whether the system is accurate, affordable and predictable.

Key takeaways:

• Agentic RAG pays for itself on multi-hop, ambiguous and "the first query used the wrong words" questions. On simple lookups it only adds latency. Route simple questions to a fast path.
• Relevance is not sufficiency. A passage can be on-topic and still not answer the question. Define sufficiency as named information slots, each backed by evidence.
• Every loop needs hard limits: max steps, max tokens, wall-clock timeout and a no-progress detector. "The model will know when to stop" is not a stopping criterion.
• Carry provenance (document, chunk, version, source system) through every step in an evidence ledger. The final answer may only cite what is in the ledger.
• Evaluate the trajectory, not just the answer: steps per question, grader accuracy, abstention correctness, cost per answered question.