Test citations for support and freshness

A link beside an answer is not proof that the answer is grounded. The cited page may discuss the topic without supporting the claim, use an obsolete version, or be inaccessible to the person expected to verify it.

The unit of review is the claim-to-evidence relationship, not the number of links. Ten references can all miss the decisive statement. One source tied to an exact version and clause may let a reviewer confirm or reject the answer quickly.

Audit the claim, not the paragraph

Break the answer into claims that would change a decision. For each claim, record:

1. the exact supporting passage;
2. whether the passage entails the claim or merely resembles it;
3. the source owner and publication status;
4. the effective or updated date;
5. the requester’s permission to open it;
6. any qualification lost in the summary.

Mark a claim as supported, partially supported, contradicted, stale, or inaccessible. “Citation found” is not a sufficient status.

Identify claims that require verification first: numbers, dates, eligibility conditions, causal statements, comparisons, policy requirements, and recommendations that trigger action. Transitional prose may not need a citation. Do not hide several verifiable claims inside one long sentence and attach an ambiguous link at the end.

Use a minimal-rewrite test for support. Could a careful reviewer, reading only the cited passage, write a statement with the same scope? If the answer has added an entity, period, geography, qualification, or causal relationship absent from the passage, mark it partial or unsupported.

Example: leave-policy answer

The answer says contractors receive ten days of paid leave and cites a benefits page. The page names ten days for full-time employees; contractors appear in a separate exclusion. The citation is real, current, and topically relevant—but it contradicts the answer.

The right fix is not a prettier citation format. The retrieval test must preserve worker type and the answer validator must compare the claim with the quoted clause.

Example: market brief

A brief states that a market grew last year and links to a report published three years earlier. The report contains a forecast, not an observed result. Another source uses a different definition of the market.

The audit labels the claim stale and definition-mismatched. The answer should either find current evidence or state that the requested conclusion cannot yet be supported.

Add deliberate traps to evaluation

Include documents with similar titles, superseded versions, pages that cite one another, and passages where the key exception appears below the obvious sentence. These cases reveal whether the system follows evidence or only keyword proximity.

Check authority separately from freshness

Newest does not mean most authoritative. A recent commentary can rank below an older but still effective policy. A standard with an early publication date may remain current until formally replaced. Record publisher, document class, effective date, retirement date, superseding relationship, and last verification date; then establish priorities for the task.

Freshness belongs to the claim. Flight status may age in minutes, an internal procedure may change only with a versioned release, and a stable definition should not fail merely because its page has not been edited recently. Avoid imposing one site-wide age limit on every source.

When no update information exists, do not silently assume that the page is current. Mark it for verification and send it to the source owner. If two authoritative records conflict and neither is retired, expose the conflict or stop instead of asking the model to manufacture certainty.

Copy this citation-audit record

Field	What to capture
claim_id	Stable identifier for the consequential statement
Claim text	Entity, scope, period, and qualification intact
Supporting passage	Smallest sufficient context from the source
Support status	Supported, partial, contradicted, or absent
Authority class	Primary policy, official record, commentary, and so on
Version status	Current, superseded, unknown, or future-effective
Accessibility	Whether the intended reader can inspect it
Review action	Keep, narrow, replace evidence, or remove

Create separate records when one claim uses multiple sources; do not compress supporting, conflicting, and background material into one label. At the end, find citation orphans—links that appear in the answer but support no identified claim.

Decide the response to weak evidence

Define whether the workflow should omit the claim, lower confidence, show competing passages, or ask for review. The fallback belongs in the product behavior, not in a hidden evaluation spreadsheet.

Common questions

Does every sentence need a reference? No. Prioritize claims that change decisions, can be externally checked, or age over time. Place the citation beside the content it supports.

Is an accessible link automatically verifiable? No. The reviewer needs the relevant passage and version. A home page, search result, or permission-denied destination does not enable verification.

Which source wins when several disagree? Apply the authority hierarchy and effective state defined for the task. If those cannot resolve the conflict, preserve the alternatives and their scope rather than synthesizing false certainty.

Boundary

This audit checks support and freshness; it does not establish that a source is objectively correct. High-stakes work still needs domain authority, provenance controls, and an accountable reviewer.