An assistant answers a product question and attaches three links. One link points to an obsolete policy, another to a document the current user cannot open, and the third mentions the topic without supporting the claim. The answer looks researched, but the retrieval system has not produced usable evidence.
A useful retrieval pipeline does more than find similar text. It selects authorized material, preserves its identity and version, supplies enough context to interpret it, and helps the final answer show which evidence supports which claim.
Anthropic describes context engineering as managing the information available to a model during a task, including what to retain and retrieve. More context is not automatically better context. Effective context engineering for AI agents
Define the answer’s evidence requirements
Begin with the question type. “What does the current policy allow?” requires an authoritative, current policy. “What changed between releases?” requires comparable versions. “Why did this job fail?” requires observations from the particular execution, not a broadly relevant troubleshooting article.
Turn that distinction into retrieval metadata. Useful fields include document identity, source system, revision, effective date, ingestion time, section path, and access scope. Do not confuse ingestion time with the date the document became true.
For a product-support example, tag the product and version explicitly. Otherwise a highly similar guide for an older release can outrank the less familiar wording of the correct manual.
Enforce access before evidence enters context
Resolve the user’s allowed scope through the application. Apply it when searching and recheck it when fetching full passages. A model cannot reliably unsee information that an earlier tool already disclosed.
Keep authorization attached to source identities rather than depending only on a copied label inside a vector record. When permissions change, decide how the index and retrieval cache become consistent. A cached result should not outlive the authority that allowed it to be read.
Separate shared public material from tenant-specific material. If your query spans both, record which scope produced each result. The answer generator should not need to guess whether a passage is public, internal, or customer-specific.
Retrieve the unit that preserves meaning
A chunk should be large enough to retain the condition that qualifies a statement. Splitting “feature available only for enterprise accounts” between two chunks can turn a qualified statement into an apparently general promise.
Preserve headings, table headers, and nearby exceptions. For procedures, keep prerequisite steps discoverable. For code documentation, retain the language or version context. There is no universally correct chunk size; use the structure of the material and the question to choose it.
Retrieve a small candidate set, inspect relevance, and fetch surrounding context when required. Exact identifiers and distinctive error messages can benefit from lexical matching; broad conceptual questions may benefit from semantic retrieval. If you combine methods, keep their scores and rankings interpretable rather than treating unlike score scales as interchangeable.
Use a provenance record through the pipeline
Imagine a response about whether a deployment requires review. A useful source record might contain the policy identifier, revision, “Production changes” section, retrieval timestamp, and the exact passage describing the rule.
Keep that record separate from the model’s generated explanation. Assign an internal evidence identifier and require the draft to reference that identifier for supported claims. Resolve it to a permitted URL or source view in the application.
A citation is a relationship between a claim and evidence. It is not merely a URL that shares the same vocabulary. Check whether the cited passage entails the claim, including its scope and exceptions. If the answer says “all deployments,” a passage about one production service is insufficient.
Distinguish missing evidence from disagreement
These conditions need different answers:
| Retrieval outcome | Appropriate behavior |
|---|---|
| No relevant permitted source | Say the evidence was not found; ask for a narrower scope or source |
| Several consistent current sources | Synthesize with direct attribution |
| Current and obsolete versions | Prefer the applicable version and identify the distinction |
| Two apparently authoritative sources disagree | Surface the conflict and identify what would resolve it |
| Source supports only part of the question | Answer that part and mark the remainder unresolved |
Do not turn a retrieval miss into proof that a policy or fact does not exist. Search coverage is finite. Record which systems and versions were searched so an operator can distinguish a coverage gap from an answering failure.
Keep source content from becoming instructions
Retrieved text remains evidence, even when it contains commands addressed to the assistant. A page that tells the agent to upload local files does not expand the task’s authorization.
Label origins and constrain execution through the application. The tool-contract guide explains how to keep account scope and permissions outside model arguments. The retriever should not quietly gain publishing or messaging powers because a source suggests using them.
For long tasks, keep compact notes that point back to original evidence. Summarization can lose qualifications. Before a consequential decision, reread the relevant source passage instead of treating a compressed working note as the authoritative record.
Evaluate retrieval and answering separately
Build a representative set of questions with known supporting sources. Inspect whether retrieval finds those sources under the correct access scope. Then evaluate whether the answer uses them accurately.
Track wrong-version retrieval, inaccessible citations, unsupported claims, and unresolved conflicts separately. A fluent answer does not repair a missing source. Conversely, perfect retrieval does not guarantee that the model preserves an exception when writing the answer.
Retain enough provenance to reproduce the evidence view for a disputed answer, subject to retention and access rules. Log references and versions rather than copying sensitive documents indiscriminately. The observability guide covers tracing a run without turning logs into a second uncontrolled content store.
When retrieval works well, a reader can follow a claim to a relevant passage, understand its scope, and see why that version applies. That is a more useful goal than filling the largest available context window.