Diagnose an AI answer before changing the model

A bad answer is evidence, not a diagnosis. Switching models after every failure hides whether the defect came from input, retrieval, instruction, validation, or the action that followed the answer.

The objective is not to invent a persuasive explanation for the error. It is to form a falsifiable cause: if this layer alone changes while everything else stays fixed, does the failure disappear? Until that question can be tested, the cause has not been isolated.

Start at the first incorrect step

Save the complete trace for the failed case: user input, selected sources, assembled context, model response, tool calls, validators, and final user-visible result. Find the earliest point where actual behavior diverged from expected behavior.

Then classify the failure:

1. Missing input: the required fact never entered the workflow.
2. Wrong evidence: retrieval selected irrelevant, stale, or unauthorized material.
3. Instruction conflict: two rules pull the answer in different directions.
4. Capability limit: the model cannot perform the transformation reliably even with good input.
5. Output contract failure: the answer contains the right idea in an unusable shape.
6. Handoff failure: a correct response is lost, altered, or misapplied by the next step.

Example: the wrong policy answer

An assistant recommends a reimbursement limit from last year. The response is fluent and follows the requested format. The trace shows that search ranked an archived FAQ above the current policy.

Adding “use the latest information” to the prompt treats the symptom. The corrective action belongs in source status, retrieval filtering, and a freshness check. The model answered from the evidence it received.

Example: a malformed tool call

An agent identifies the correct customer and correct action but sends a date in the wrong format. The downstream API rejects the request, while the interface reports “customer not found.”

The reasoning was not the earliest failure. The schema validator and error translation were. A larger model may still produce the wrong format, and it will not repair a misleading error message.

Match the fix to the layer

• Missing input → change collection or ask a clarifying question.
• Wrong evidence → change source selection, ranking, or freshness controls.
• Instruction conflict → establish precedence and remove duplicated rules.
• Capability limit → narrow the task, add deterministic computation, or compare models.
• Contract failure → add structured output and validation.
• Handoff failure → repair schemas, state, retries, and error reporting.

Test the diagnosis with the smallest change

Freeze the original case, timestamp, workflow version, model configuration, and dependency responses. Replay a copy while substituting only the suspected layer. For stale evidence, keep answer generation fixed and supply the correct source. For a possible capability limit, preserve the input and acceptance contract while comparing candidates. For a suspected handoff defect, bypass the model and submit a hand-written contract-compliant payload.

Change one variable per experiment. Replacing the model, rewriting instructions, rebuilding the knowledge source, and altering the parser in one release may improve the result, but it destroys the evidence needed to learn which intervention worked or introduced the next problem.

Do not retest only the famous failure. Include an ordinary case from the same slice and a neighboring edge case. Stronger refusal wording, for example, may block one unauthorized answer while making the system reject well-supported routine requests.

Copy this failure record

Record field	Evidence to capture
Expected behavior	Numbered acceptance rule and a passing example
Observed outcome	What the user saw or the system actually executed
First divergence	Earliest trace event that broke the expectation
Cause hypothesis	Statement that a single-variable test can disprove
Minimal experiment	What stays fixed and what alone will change
Missing safeguard	Validator that should have stopped propagation
Repair scope	Affected slices, owner, and regression cases
Closure condition	Evidence required to declare the defect resolved

“The model hallucinated” is a weak cause because it does not explain why inputs and controls allowed the error to reach someone. A testable record says: “retrieval did not exclude archived policy, so the old revision entered context; after adding status filtering, this case and three version-boundary cases all select the active document.”

Know when a model change is justified

Treat capability as the leading cause only after confirming that input is complete, evidence is correct, instructions are consistent, and validation works—yet representative cases still fail. Compare candidates against the same snapshots and contract. Otherwise the team is comparing different experiments.

A new model that improves prose while leaving the blocking defect untouched is not a successful diagnosis. Conversely, if deterministic date normalization removes malformed tool calls, a more expensive model should not own work that software can guarantee.

Common questions

What if the full trace was not saved? Stop tuning from memory. Add the minimum instrumentation needed to retain input, retrieved evidence, versions, tool calls, and error codes on the next occurrence. Mark the historical case “cause unconfirmed.”

Where do we begin when feedback only says “bad answer”? Rewrite it as an observable difference: which field is missing, which claim lacks support, or which action should not have occurred. Then locate the first trace event that produced that difference.

What if the failure has several causes? Separate the initiating cause, contributing conditions, and failed safeguards. Test them independently and start with the earliest layer that affects the widest slice.

Boundary

One failure may have several contributing causes. Record the earliest cause separately from later safeguards that should have caught it. The purpose of the tree is not to assign blame; it is to stop expensive changes at the wrong layer.