When RAG lacks evidence, should it answer, clarify, or hand off?

A search result is a candidate source, not permission to make a claim. It may cover the wrong product revision, omit an exception, be too old, or be unavailable to this requester. A RAG assistant therefore needs a decision before it writes the answer: is the evidence sufficient for this exact question, and what should the user see if it is not?

This page designs that response gate. The retrieval trial diagnoses what the search stage found; the index refresh trial checks whether a source change reached the serving path; the citation audit inspects claims after generation. Here the output is a routing rule and test cases for answer, clarify, say unable to verify, or hand off. Microsoft’s RAG evaluators distinguish retrieval quality, groundedness (no claim beyond supplied context), and response completeness (no missing critical information; its evaluator is in preview). Passing one of those checks alone does not choose the right user response.

Check the evidence for the requested claim

Freeze the question, caller identity and permitted scope, relevant product or jurisdiction, requested time, corpus version, index snapshot, and source authority rules. Break a multi-part question into claims whose truth could change the user’s decision. For each, record the smallest passage that supports the exact entity, period, condition, and exception. A plausible top hit or a model’s confident wording is not that passage. Google’s grounding overview says linking output to verifiable sources reduces the chance of invented content; it does not promise that retrieval makes every answer true.

Classify the gap before choosing an action. A missing user-supplied revision is different from a missing source. A source may be present but incomplete, contradicted by another effective source, withdrawn, not yet visible in the index, or outside the caller’s permission. Check authorization before a passage enters model context. Azure AI Search documents native permission filters and a separate string-based security-filter pattern. With the string pattern, the application must authenticate the caller and attach a verified filter to each query; a principal string does not itself authenticate or authorize. Restricted hit IDs, text, and even an existence signal belong only in authorized backend diagnostics or synthetic tests, never in the answer model’s context, user response, or browser workbench.

If a source is known to have changed, verify its search and cache state with the index refresh acceptance trial. Until then, a high similarity score on the old copy is evidence of a search match, not a measured probability that the requested fact is correct. Do not turn a completed search with no eligible evidence into a negative policy decision: “I cannot verify eligibility from my available sources” is different from “you are ineligible.” A retrieval timeout or tool error is a separate state: the system has not completed the check. Retry only within a stated time and attempt limit, then say it is temporarily unable to check or seek a confirmed human handoff. It cannot infer that no source exists.

Choose a route for each claim

Route	Gate before using it	What the user should receive
Answer	Every claim actually answered has sufficient, current, authorized support. Answer the entire request only when all consequential requested claims pass. A partial answer is allowed only for separable claims whose omission cannot change the meaning or safety of the supported part.	Cite each supported claim. Name each unanswered part separately and give it a clarify, unable-to-verify, or human route; never present a partial answer as the complete decision.
Clarify	A specific missing fact from the user would select between already documented branches.	The smallest question that changes the route, plus why the answer depends on it. No provisional guess.
Unable to verify	A completed permitted retrieval and source check still leave a non-consequential fact unsupported, stale, or unknown.	State that the fact cannot be confirmed from the available materials, name the checked scope when safe, and give a realistic next step. Do not assert the opposite fact.
Human handoff	Evidence conflicts without an approved priority rule, the request would trigger a consequential action, an authority decision is required, or a deadline makes waiting unsafe.	Make no automated decision. Claim the case was handed off only after an actual queue receipt names an owner and next update. If acceptance fails, give a truthful approved contact route without claiming a transfer.

These are workflow rules to test, not a universal score threshold. For a multi-part request, assign a route to each claim and compose one response with the unresolved parts visible. If omitting a part could mislead the user into acting, the hard stop governs the whole response. Hard stops for authorization, source conflict, and high-impact actions take priority. A numeric relevance score may help investigate a miss, but it cannot certify claim support. The OpenAI Cookbook includes a bounded question-answering example that instructs the model to say “I don’t know” when the provided article lacks the answer; a production route also needs to decide when a clarification or accountable handoff is more useful.

Example one: the manual has two hardware revisions

In this fictional support exercise, the approved AeroSensor M2 Rev A manual says its export needs a network connection. The approved Rev B manual describes offline CSV export over USB. Both are current for their respective hardware revisions. A customer asks, “Can my M2 export CSV offline?” Search returns both manuals, but the customer has not supplied the revision on the device label. Answering “yes” from the Rev B hit would turn an unresolved branch into a false universal statement. The gate asks: “Which hardware revision appears on the device label? The manuals give different answers.”

If the customer replies “Rev B,” the assistant can answer for Rev B and cite the exact manual section. If the question also asks for a maximum CSV file size and the manual gives none, it can state the supported export method and separately say that the size limit cannot be verified; the latter must not be buried in an apparent full specification. If they reply “Rev C” and no current Rev C manual is available, asking the same question again does not repair the source gap. It should say that offline export for Rev C cannot be confirmed from the approved manuals it can use and point to the product support owner. It must not infer Rev C behavior from either older revision. This case distinguishes a clarifiable input gap from an evidence gap after clarification.

Example two: a safety fault needs an accountable owner

In another fictional exercise, a building operator asks whether to restart a ventilation controller after fault E17. The current authorized manual has routine restart information but no procedure for E17; a highly ranked bulletin containing that code has been withdrawn. The assistant cannot splice the routine procedure into a new one or cite the retired bulletin. Because the request would guide a physical action, it first blocks an automated recommendation and requests human intake. Only if the queue confirms acceptance, the assigned safety engineer, and a next-update time may the user message say: “I cannot verify an approved restart procedure for E17 from current material, so I am not recommending a restart. Case [receipt ID] was accepted by [assigned engineer]; your next update is due at [confirmed time and time zone].”

The authorized backend’s handoff record includes the request time, fault code, permitted manual revision, search and index versions, and a categorical note that an ineligible hit was blocked. A restricted or withdrawn hit ID remains in access-controlled diagnostics only; it is not put into model context, the browser worksheet, or the user-facing packet. If intake fails, the assistant says the case was not transferred and gives the established emergency or on-call contact path. If an accepted owner cannot respond by the promised time, the queue must show the delay and update the user; silence or an automatic restart recommendation is not a fallback.

Test both wrong answers and needless abstention

Create labeled cases with complete support, a missing user variable, an absent source, a separable partial answer, a stale or withdrawn hit, conflicting effective sources, a synthetic permission-filter negative control, a retrieval outage, and a consequential action request. Keep any real forbidden-hit diagnostics inside the trusted backend; the answer model and browser worksheet receive no forbidden ID or existence flag. A domain reviewer assigns the expected route and allowed wording before tuning prompts or any score threshold. Hold back some cases for a final check. For each run, compare the actual route, claim-level support, clarification question, user-visible limitation, and confirmed human receipt against that record.

Report denominators by case type: unsupported answers on no-evidence cases, unwarranted “cannot verify” replies where enough evidence existed, clarifications that asked for an irrelevant fact, claimed-but-unaccepted handoffs, wrong treatment of a retrieval error as source absence, and access disclosures. A low unsupported-answer count achieved by declining every question is not a successful assistant. Recheck after a source, permission, index, model, or policy change, and include peak human queue time when handoff is part of the promise.

Fill the no-evidence response gate

Use the workbench below to record a synthetic query ID, eligible source IDs, claim-by-claim evidence states, user wording, accepted handoff status, and held-out results. It saves a draft in this browser, so leave real user questions, forbidden source IDs, and restricted-text diagnostics in controlled backend records. Another reviewer should be able to replay the same sanitized case under the same authorized caller scope and explain why each route was allowed.

Boundary

This gate selects the next response for a defined RAG workflow. It does not certify that an underlying source is true, grant access to material the caller cannot see, or authorize a consequential action. If source authority or the underlying policy is disputed, resolve it with the responsible owner rather than letting the model decide which document wins.