How to implement an AI agent tool-calling loop

An agent that works is not “the model suggests a next step, so the program does it.” The application owns the loop: request a model turn, interpret a complete response, validate its tool requests, execute authorized actions, return each result in the correct protocol, and decide whether another turn is warranted. A mismatch anywhere can feed the model the wrong observation—or mistake an unfinished action for a failure.

This guide covers the state round trip to implement or review in a custom tool-calling loop. It does not require recording hidden reasoning. Review the visible tool request, executor decision, external effect, and result. If a tool’s parameters, authority, and repeat behavior are not defined yet, start with the tool-calling contract. The runtime loop guard governs whether work continues after each observation.

Decide who owns control of the loop

Whether you use ReAct, function calling, or an agent SDK, the application needs to know where a turn begins, which response state means “tools are next,” who may approve a write, and where the run is saved when it stops. A framework may manage a message protocol or schedule work, but it cannot choose an organization’s authority policy or guarantee that a remote write can be looked up.

Model the run as a finite set of states instead of an unconditional while True loop:

Current event	Controller’s next move
Complete final response with no tool request	Check the task’s acceptance condition and save a terminal state
One or more complete tool requests	Save the assistant request, then normalize and validate each call
Refusal, truncation, or request failure	Save its type; retry, degrade, or hand off by policy; never execute partial arguments
Human approval required	Save the pending action and bound arguments; pause automation
Tool clearly succeeded or was rejected	Save the result against its call identifier using the required protocol
Remote write outcome unknown	Query the authoritative system; pause if still uncertain rather than replaying

Keep “model request” and “tool execution” as separate trust boundaries. The model proposes an action. The runner checks caller identity, object scope, arguments, business preconditions, and current approval. Authorization should never live only in the natural-language tool description.

Put provider-specific conversation shapes behind an adapter

Providers represent a tool round trip differently. The OpenAI Responses API uses a function_call and its related call_id, followed by a function_call_output. Chat Completions represents assistant tool_calls and matches each result through tool_call_id. The Anthropic Messages API uses a tool_use content block and a matching tool_result, with provider-specific stop states for the handoff. An SDK may wrap some of these fields or states for you.

An adapter can normalize them to internal events such as assistant_tool_request, tool_result, assistant_final, and paused_for_approval. Persist the provider, API version, original response identifier, and enough of the original call structure to reconstruct what happened. When sending the next turn, the adapter must restore the role order and correlation fields expected by that API. A home-grown generic JSON envelope is not a substitute for the provider’s required message sequence.

A safe round trip looks like this:

1. Save the turn and model response. Confirm it completed according to the provider’s contract and that its arguments are no longer stream fragments.
2. Parse each tool name and argument set. Check the allowlist, schema, business conditions, caller identity, resource scope, and approval.
3. Run authorized tools and save succeeded, rejected, ordinary failure, or unknown outcome for every call ID. Pause when approval is required.
4. Persist the original assistant tool request with its matching results, then build the next conversation using the provider adapter.
5. Request another model turn only if current results allow it, budgets remain, and no human action is pending.

If one assistant response contains multiple calls, do not return only the successful ones. Give every call ID an explicit result. A rejected request can receive a structured error, but it must not run. Parallel execution is suitable for independent reads, or for operations whose interaction has already been designed and tested. Writes against the same record and dependent actions should run in a deliberate order. Completion order can differ; results still have to match by call ID.

Assemble a streamed call before executing it

A stream may split the tool name, JSON arguments, or call identifier across events. The first fragment is not a complete authorized request. Accumulate fragments by the provider’s event index or call ID; wait for an explicit completion event; then verify that the final arguments parse and contain required fields. If the connection ends while arguments are partial, persist an incomplete state and stop that turn.

Distinguish a refusal, final answer, tool handoff, model timeout, and transport interruption. Provider fields differ, so one universal finish_reason check does not exist. Read the completion status and stop reason defined by the selected API. If an SDK gives you a normalized result, test that behavior against the version you ship. A plausible-looking fragment from a truncated response must not trigger a side effect.

Match results by call identifier, not array position

Each proposed tool action has its own correlation identifier. A runner may call two safe read tools concurrently, but it cannot put the weather result where the train result belongs just because the weather request finished first. Store the request summary, tool version, authorization decision, execution result, and timestamp under the call ID. When a run resumes, reconcile that ID and the authoritative target state to determine whether an action has completed.

Persist the model request, execution result, and conversation context as one run state. A process can crash between “the tool ran” and “the result was written back.” Reads may be safe to repeat under their contract; writes need a target-system lookup or an idempotency mechanism that the downstream service actually enforces. Do not ask the next model turn until call state is saved, or a restart could produce the same external effect again. Long runs must also carry approvals and cumulative usage forward; see the long-task resume-state contract.

This pseudocode separates the responsibilities: adapter owns provider protocol, while executor owns server-side authorization and execution.

save(user_event)
for turn in remaining_turn_budget:
    response = adapter.request(history, allowed_tools)
    event = adapter.read_complete_event(response)
    save(response, event)
    if event is final:
        return validate_and_present(event)
    if event is incomplete_or_refused:
        return pause_or_handoff(event)

    calls = validate_batch(event.calls, identity, permissions, approval)
    if calls.need_human_approval:
        return save_and_pause(calls)
    results = executor.run_independent_calls(calls)
    save_results_by_call_id(results)
    history = adapter.append_results(event, results)
return save_and_handoff("turn budget reached")

“Batch” does not mean a database transaction. If some calls can run and others need approval, state whether partial completion is permitted. A group of side-effecting calls should usually pause for replanning or human approval instead of quietly executing its first half.

Example: parallel travel-status lookups

A travel assistant must answer whether a train is delayed and what the weather will be at its destination. The model requests get_train_status and get_weather. The runner first verifies that the train number and date are inside the user’s authorized scope, then queries both read-only tools. The weather service can return first without changing either call’s identity. On reply, the weather observation goes to the weather call ID and the train observation to the train call ID. If weather times out, the verified train result can still be returned while weather is reported unavailable; it must not be attached to the neighboring call.

Test reverse completion order, one failed tool, an invalid train number, and a truncated response. Acceptance evidence is that every state in the final answer matches the correct call ID and original response.

Example: a workshop reservation with approval and timeout

An event assistant is asked to reserve a workshop seat. It first uses a read-only tool to confirm availability and eligibility, then proposes the reserve_seat write. The runner verifies the signed-in identity, event, participant authority, and whether the approval still covers those exact details. If an event owner must approve, save the pending record with the event and participant bound to it. Do not send the model off to propose another write while this one waits.

After approval, the reservation service accepts the write but its response times out. Mark the state unknown and look up the operation in the reservation system. If one reservation is confirmed, return that result. If the lookup remains inconclusive, pause for the owner to verify. Include a “booking succeeded but its response was lost” test so a resumed turn cannot reserve the seat twice.

Turn protocol behavior into regression cases

Cover at least: a final answer with no tool, a single call, multiple calls returning out of order, an unknown tool, invalid arguments, a truncated stream, a tool error, approval pause, executor rejection, interruption after a write, process recovery, and model refusal. For every case, record expected state, actual execution count, result correlation, and whether an external effect occurred. Re-run the suite when the API, SDK, persistence format, or tool behavior changes.

Retain only the data needed to debug: run ID, response ID, tool name and version, argument summary, policy decision, call ID, external state, and result code. Do not put credentials, full personal records, or hidden reasoning in general logs. Use an event-level trace review to find the first mismatch among request, execution, and result when something goes wrong.

Common questions

Do I still need to understand these states when using an agent SDK?

Know what the SDK manages, which executions persist, and how retries, pauses, approvals, and errors are represented. Let an SDK handle message exchange if that is useful, but still test business permissions and external effects. Keep acceptance cases for the protocol behavior of the version you deploy.

Should the model’s thinking be stored in a trace?

Hidden reasoning is not required. Visible tool requests, executor checks, tool observations, approvals, and terminal reasons are usually enough to debug the protocol. Decide separately whether user input or tool results may be retained, and apply redaction and access controls.

Boundary

The pseudocode describes loop boundaries and is not production-ready as written. Provider message fields, refusal states, stream events, and tool-result formats change; implement adapters from the current API and SDK documentation. A loop also cannot replace tool authorization, downstream idempotency, sandbox isolation, human approval, or a separate incident-recovery plan.