When should you compact a long AI conversation?

A long task can accumulate corrections, rejected ideas, approvals, and pages of tool output. Keeping every turn consumes the model’s working window. Deleting the oldest half may silently turn “not approved” into “agreed.” The practical question is when to change the form of old history, what must survive, and how to test the next turn after the change.

The context map inventories the instructions, evidence, examples, and history assembled for a call. This guide starts later, when a running task needs a compaction checkpoint. A generated summary does not become a source of authority: the application still supplies its current rules, actual permissions, and access to original evidence.

Trigger from the next turn’s needs

Save the request actually sent and the token usage reported by the provider. Check the selected model and API’s context window. Estimate the next user input, possible tool result, and space needed for the answer. System instructions, tool definitions, documents, images, previous turns, and generated output all compete for that window. Cached input still occupies it even if the provider charges a different rate for those tokens.

Use the per-call context budget to test that next request against its separate input, output, and window limits before deciding how much history must change.

Compact at a safe turn boundary when the projected next request may exceed the available space, or when old, no longer relevant material is obscuring the current decision. Do not publish a universal “compact at 80%” rule: a short reply and a large tool result require different reserves. An on-demand compaction call itself must fit within the provider’s window, so plan before the transcript has already overflowed. Some APIs also offer automatic threshold compaction; verify the exact behavior for the API in use.

Before changing history, keep an original snapshot with its owner, model configuration, and source locations. Put redacted references in this browser worksheet; keep raw transcripts and sensitive documents in the team’s approved storage.

Give each piece of history a destination

Destination	Good candidate	Check before using it
Keep recent turns verbatim	A fresh correction, exact wording, or open condition	How much of the new window will they occupy?
Summarize as typed state	Settled decisions, rejected proposals, open questions	Are negations, numbers, owners, and status preserved?
Archive for retrieval	Long contracts, search results, logs, and cited passages	Can the same version be fetched under the requester’s permissions?
Drop from active context	Repeated, superseded, reproducible output	Did it contain a decision or an anomaly not recorded elsewhere?

For each consequential item, record its turn or file, claim, status, destination, and source pointer. Distinguish confirmed, proposed, superseded, and rejected. An approved annual cap of $180,000 and a later proposal to raise it to $200,000 must not become “roughly $180,000–$200,000.” If the next answer requires a clause citation, a summary can point back to the clause; it cannot stand in for the clause. If the original cannot be retrieved, say that the claim lacks verifiable support.

Keep tool calls and their results together at the cut. Anthropic’s on-demand compaction, for example, rejects a request ending with an assistant tool call that has no result. For an external write with an uncertain outcome, retain the operation ID, authoritative lookup route, and unknown status in application state. Do not let a conversation summary invent success or failure.

Know what can be inspected

Provider implementations differ. OpenAI Responses server-side compaction can emit an encrypted compaction item at a configured threshold; its documentation says the item is opaque and not meant for human reading. An audit therefore needs an independent, visible state ledger and behavior tests, not a claim that somebody reviewed every word inside the item. Anthropic on-demand compaction returns readable text with a signature. Its signed block must be passed back as returned, rather than edited in place. If the default summary omits a needed fact, choose a different cut or summarization instruction, regenerate, and retest.

Clearing old tool results is another operation. It may reclaim space without producing a useful account of what the results proved. Preserve the error signature, decision, and route back to the original before clearing the bulk output. A model’s statement that something “was confirmed” never replaces an approval record, source revision, or authoritative tool result.

Compare before and after on the same questions

Before compaction, write a few questions likely to expose loss. Save the expected answers and their original locations. Ask the same questions afterward with the same model configuration and inspect both the answer and any proposed action. Include at least these probes:

1. Exact boundary: Does the result retain an amount, date, “must not,” and “not yet approved” accurately?
2. Conflicting history: Does the latest correction supersede an old decision, while a proposal remains only a proposal?
3. Evidence: When asked for exact wording, does the system fetch the right revision instead of citing its own summary?
4. Next action: Do a pending tool result, human approval, or named owner still prevent a premature step?

Lower token use alone is insufficient. Compare accepted answers, review effort, and latency for the task cases that matter. If a critical probe fails, restore the original snapshot, keep a longer verbatim tail or summarize a smaller range, then repeat the same probes. Leave high-impact actions with an owner until the checkpoint passes.

Example: a supplier quote loses two prohibitions

A buyer and assistant review several suppliers. At turn eight they confirm an annual cap of $180,000. At turn nineteen the buyer rejects automatic renewal. At turn thirty-one they say Supplier B has not passed security review and may not receive a purchase order. Many product details and draft emails follow. A short summary that says only “budget about $180,000; B is preferred” makes the next step unsafe.

The checkpoint records the cap, no automatic renewal, and no approved order for B as separate state rows with turn and approval-record pointers. Repetitive brochures leave active context; the current quote stays retrievable by version. The paired questions ask whether an order may be sent to B, whether renewal may be automatic, and what cap applies. If any response upgrades a proposal into permission, restore the snapshot and revise the cut.

Example: incident investigation keeps an unverified fix

A support engineer uses an assistant to investigate intermittent API timeouts. Earlier turns contain thousands of near-duplicate log lines. The useful live state is an error signature, affected route, eliminated hypotheses, and a patch that has been submitted but not yet regression tested. The logs remain in controlled storage under an event ID. The summary carries the signature and elimination evidence; the latest patch discussion stays verbatim.

If a test tool call is still open, wait for its result or pause compaction and hand the pending action to the owner. The after-compaction probe asks, “Has the fix been verified?” A valid answer says no, names the remaining test, and gives the log location. “The timeout is resolved” fails even if the response is fluent and cheap.

Boundary

A short exchange rarely benefits from compaction. If the real failure is wrong evidence entering the request, return to the context map first. If the goal, information access, or approval scope changes, establish new task state rather than carrying old authority through a summary. Long-lived cross-session preferences and deletion requests need a separate memory lifecycle. Compaction cannot replace original evidence, backend permission checks, or a person’s review of consequential decisions.