How many tokens should one LLM call reserve for input and output?

A team estimates that a request is “well below the model’s million-token window” because its visible prompt is short. The assembled call also contains system rules, tool definitions, earlier messages, a retrieved PDF, and the result of an account lookup. The answer needs room for generation too. How much of this particular call can safely enter, and how much must remain available before the first output token?

The context map identifies what the model receives and why. This page measures the next assembled request and makes a go, reduce, or route decision against the exact model and API limits. The compaction guide decides how to preserve an already long history; this budget must be checked even for a first call. The worksheet below records one call or one tool-return continuation at a time.

Record three ceilings before assigning any tokens

For the chosen model snapshot and API, record its context window, any distinct maximum input that the model or API explicitly specifies, and its maximum generated output. Also record the generation cap actually configured for this request. These are different quantities. A model reference may state the window and maximum output without a separate input limit; do not infer one from the window or treat the window as the length of a visible answer.

Use a conservative planning check:

counted next-request input I + configured full-generation cap G + headroom H ≤ context window W

Also check I + H ≤ model input limit where a separate limit is explicitly specified, and G ≤ model output limit. G covers everything the model may generate in this call: visible text, tool-call structure, reasoning or thinking, and other non-visible formatting. Set G from measured full-generation use, including a long valid case. H covers uncertainty in input counting and growth between counting and dispatch, so include it in the separate input gate too. Choose it from credible input variation and observed counting differences, then write down the reason. A fixed percentage for every provider and task would hide the cases that need more space.

This is a release gate you set, not a claim that every API rejects requests using the same formula. Anthropic, for instance, documents different overflow behavior across model generations. If the input alone exceeds the window, it is invalid; newer Claude models may accept an input plus max_tokens value above the window and stop only when generation reaches it. Plan for a complete answer rather than relying on that late stop.

Count the request that will actually be sent

Build the candidate payload first. Include instructions, current user message, retained history, examples, tool names and schemas, retrieved text, images and files, output schema, and any tool results already returned. Count this assembled payload against the intended model. Counting only the user’s prose misses structural and multimodal tokens. For an application-side client tool, its returned result enters the next model call. Before it runs, estimate that continuation’s retained input, an upper bound for the tool result, and continuation structure; check projected I + H against any explicit input limit and projected I + G + H against W. After the result arrives, assemble and count the actual next request again. A provider-hosted server tool may return and continue within the same interaction instead, so bound its calls and result size and inspect actual usage rather than assuming a separate next request.

The provider counters need different reading rules:

API documentation checked	Pre-send count and what it does not settle	After-call reading
OpenAI Responses	input_tokens.count accepts the same request structures as Responses, including text, tools, images, files, and conversations. Its count includes representation such as message roles and boundaries. It cannot predict a tool result that has not arrived.	usage.input_tokens includes cached input as a subset; usage.output_tokens includes generated tokens beyond visible text.
Anthropic Messages	count_tokens accepts system, client tools, images, and PDFs, but its number is an estimate that may differ slightly from actual use. The endpoint rejects several server-tool, MCP connector, and URL/file-source combinations; use response usage for those paths.	With caching, total input is input_tokens + cache_read_input_tokens + cache_creation_input_tokens, not input_tokens alone.
Gemini Interactions documentation	count_tokens reports input only. The actual interaction reports input, output, thought, cached, and tool-use counters. Check the current model’s documented limits.	total_output_tokens and total_thought_tokens are separate; add at least these two to evaluate full generation, and inspect tool usage separately. A cached subset is not extra input.

Prompt caching changes processing cost and often latency, not the amount of material the model can reference. Anthropic explicitly says cached reads and writes still occupy the window. OpenAI reports cached_tokens inside total input usage. Keep full input tokens in the capacity column and put cache hits in a separate cost column; use the cache-hit test for whether the reuse actually occurred.

Reserve the complete generation, not just the visible answer

An output cap is not a character limit on the final paragraph. OpenAI says reasoning tokens are not visible through its API but occupy the window and count as output; its output count may also include invisible formatting. The cap can be spent before a visible answer appears. Anthropic counts thinking within its max_tokens and context window, while earlier thinking blocks may be kept or stripped on later turns depending on the model. Gemini’s current Interactions thinking documentation says max_output_tokens includes thought tokens; a cap reached during thinking can leave truncated or empty visible output. These details differ, but all make visible word count an unsafe proxy for full generation.

Start with a complete output target: required fields, two languages if relevant, citations, and any tool-call step the model must produce. Compare actual total generated usage from several representative cases, including a long but valid one. Set G high enough for the needed response and inside the model output limit. If the task cannot fit, shorten an explicitly optional field, split the response into independently checkable stages, or choose a model with suitable limits. Do not silently truncate a required JSON field or reduce reasoning space until the output happens to be short.

After each call, inspect provider status, stop or incomplete reason, actual usage, and the downstream acceptance check. A response that stops at a token limit is not a completed answer merely because its first sentences look plausible. For structured output, parse the whole response and apply the output protocol check. Use observed full-generation demand to revise G, and input count differences or pre-dispatch growth to revise H; a cache hit is not evidence that the request has become smaller.

Example: warranty lookup meets an oversized account result

A service assistant answers a fictional equipment-warranty question from approved policy excerpts. Its test model has an illustrative window of 64,000 tokens, a separately documented 56,000-token input limit, and an 8,000-token output limit. The assembled request counts 24,000 input tokens. It sets a 6,000-token full-generation cap and a 2,000-token input headroom based on prior count differences: 24,000 + 2,000 = 26,000 ≤ 56,000 for input, 6,000 ≤ 8,000 for output, and 24,000 + 6,000 + 2,000 = 32,000 ≤ 64,000 for the window.

The assistant then calls an application-side account tool. Before it runs, the team projects the continuation: 23,000 retained input tokens, an upper bound of 25,000 for the result, and 1,000 for continuation structure. Projected I = 49,000 passes the input gate with headroom (49,000 + 2,000 = 51,000 ≤ 56,000) and the window gate (49,000 + 6,000 + 2,000 = 57,000 ≤ 64,000). The tool is poorly scoped and exceeds that bound with a long event history. After adding the real result and continuation items, the new assembled request counts 61,000 input tokens. That breaches the separate input gate with headroom (61,000 + 2,000 = 63,000 > 56,000), and 61,000 + 6,000 + 2,000 = 69,000 also breaches the team’s window gate. The assistant must not send it or use the earlier preflight as approval. The tool owner changes the response to approved warranty fields and an event locator; the team builds and counts the continuation again at 27,000 input tokens. It verifies that the answer still cites the correct policy and account fact. This is a tool-result size and data-selection repair, not a reason to discard a required warranty condition.

Example: a bilingual incident brief runs out of output

An operations group needs one English and one Chinese incident brief in a strict JSON object. A synthetic test model has a 40,000-token window, a 32,000-token input limit, and an 8,000-token output limit. Its complete input counts 17,000 tokens, so input capacity is comfortable. The team nevertheless sets a 1,600-token generation cap because the final text looked short in an earlier example. A harder case generates planning and formatting tokens before both language fields are complete; the response ends incomplete and its JSON cannot be parsed.

The repair is based on actual usage, not an assumed ratio between characters and tokens. The team runs several approved synthetic cases, including the longest valid bilingual brief, and records the complete generation count. It raises the full-generation cap to 4,000 and keeps 2,000 tokens of justified input headroom: 17,000 + 2,000 = 19,000 ≤ 32,000 for input, 4,000 ≤ 8,000 for output, and 17,000 + 4,000 + 2,000 = 23,000 ≤ 40,000 for the window. It then checks provider completion status, JSON parsing, both language fields, and factual agreement. If a real case needs more than the model’s output limit, it will split the two language outputs with a common incident fact record rather than accept a partial object.

Boundary

This sheet plans capacity for one model call and repeats after every tool return or payload change. It does not decide which source should enter context, how old history should be compressed, whether a prompt prefix was cached, or what the completed workflow costs; use the linked pages for those decisions. The figures in the examples are invented model limits for arithmetic, not vendor defaults. Keep only synthetic case IDs and aggregate token counts in this browser-saved worksheet; store private prompts, documents, tool results, and API logs in approved systems.