How do you load and unload an agent plugin safely?

An agent extension can add a tool, intercept a call, watch a directory, or keep a connection open. Removing its file or disabling its setting is not enough evidence that those effects stopped. A stale approval callback can still affect a write; a watcher from an old session can still read files. Before release, test the extension as a lifecycle of capabilities and resources, including what remains after reload and failure.

The tool contract defines one callable operation, while the skill decision card asks whether instructions should be packaged for repeated work. This page addresses executable code loaded into a running agent host. The recovery runbook handles an incident after effects may have occurred; this acceptance test tries to prevent obsolete code and resources from surviving routine changes.

Do not confuse three kinds of hook

Hook names and cleanup guarantees belong to a particular framework and release. Treat the table as a documentation checkpoint, then test the exact version you deploy.

System	Documented integration	Lifecycle implication to verify
Pi extensions	An extension factory registers tools, commands, and pi.on() event handlers. The current docs say pi.on() returns an unsubscribe function, reload replaces the extension runtime, and long-lived session resources should start at session_start and close idempotently at session_shutdown.	Test that an old handler no longer affects new calls and that your own watcher, timer, or socket is closed. The docs say handler errors are reported and execution continues where possible, while a tool_call handler error blocks that tool; do not apply that rule to every event.
Cordis core and primer	The primer describes scoped registrations through ctx.effect() or ctx.on() and a disposer for each reversible side effect.	Check the actual scope and teardown order on your pinned build. Cordis core states its API is still unstable; a primer example is not a permanent contract for every host built with Cordis.
OpenAI Agents SDK for Python	RunHooks and AgentHooks expose callbacks around agent, handoff, and tool events.	These are run observation hooks. Their existence does not establish a plugin loader, hot reload, or automatic disposal of your background resources. The host application must own those behaviors.

Pi’s extension docs also say extensions run inside the Pi process with its operating-system permissions. A registry entry or callback is not an operating-system sandbox. Review source provenance, version, filesystem/network reach, credentials, and data access before loading an extension. Limit capabilities in the host and deployment environment, then test actual access boundaries.

Inventory the effects before enabling the plugin

For one version, list every registration: tool name and schema, command, event and result type, provider, UI element, and permission or approval hook. Separately list resources created by code: watcher, timer, socket, child process, file handle, queue subscription, and pending async task. Give each one an owner, creation point, and disposal action. Track whether it belongs to a process, extension instance, session, or single agent run; those lifetimes are not interchangeable.

Write a small state plan: inactive → initializing → active → stopping → disposed. This is a release-test model, not a claim that all three frameworks expose these states. During initialization, keep registrations and resources in a ledger so a failure halfway through can unwind completed steps. During stopping, first prevent new work from entering, then let active work finish or cancel it by a documented rule, unregister hooks, close resources, and inspect remaining capabilities. Cleanup should be safe to call twice.

An event listener that returns an unsubscribe function covers the listener registration, not necessarily a watcher opened inside its callback. A disposer can remove in-process registrations; it cannot retract an email already sent or undo a database write. Keep those external operations in a separate ledger with identifiers and a reconciliation or compensation owner.

Test event errors and repeated load cycles

For every event the extension handles, write down whether it is notification-only, transforms data, can block an action, or launches side effects. Record the declared return type, handler order, error behavior, and whether an in-flight dispatch can still invoke a removed handler. Do not infer the behavior from a similar event name in another SDK.

An approval gate should fail closed in the host’s enforcement path. If a callback throws, times out, or is absent after a partial load, a dangerous write must remain denied. Pi currently documents a blocking rule for a failed tool_call handler, but another event or host may only log and continue; test the specific path. An observer that merely logs can often fail without stopping the run, provided the product’s audit requirements still hold.

Use a fixed acceptance sequence with synthetic actions:

1. Start clean; activate version A. Confirm each expected capability appears exactly once and one event produces one callback.
2. Hot reload to B, then hot reload B again. Trigger the same event after each reload; confirm one active callback, current policy, and no callback or resource from A. Check an event already in flight separately, because unsubscribing may not rewrite its dispatch list.
3. Disable the extension. Confirm tools and commands are no longer discoverable or callable, no new callback fires, no watcher/timer/socket remains, and access is denied by the host even if a stale client holds an old tool name.
4. Inject an initialization failure after one registration and one resource are created. Confirm the partial instance is unwound and can be loaded cleanly later. Repeat cleanup to catch double-close errors.
5. Inject a handler error and a shutdown timeout. Confirm the declared fail-open or fail-closed result, visible diagnostics, pending-work disposition, and an owner for unresolved external effects.

Compare a before/after registry snapshot, callback counts, open-resource counts, and controlled external-operation log. A green “reload complete” message alone does not prove old authority disappeared.

Example 1: an approval hook registers twice

A finance assistant has an extension that blocks a payment-submission tool until a reviewer approves the exact recipient and amount. Version A registers the hook. A developer reloads version B to tighten the rule, then reloads again. In a flawed host integration, A’s callback remains registered and the B callback is added twice. One submission now causes three approval prompts; worse, an old callback may use an obsolete allow rule.

The test records a synthetic operation ID, the callback version and instance ID, and the final decision. It expects exactly one current policy decision and a backend authorization check before any write. Make B’s hook throw and confirm the payment stays blocked. Disable the extension and try the same tool name: the host must not silently drop the approval gate while leaving the payment capability callable. If disabling the guard requires disabling the write tool too, make that dependency explicit in the host configuration.

No test sends a real payment. If a write had already crossed the boundary before a reload, removing the hook cannot reverse it; the operation ledger and recovery rule determine the next action.

Example 2: a file watcher survives the session

A research extension watches an approved input folder and publishes a “new source available” event. It starts a watcher when a session begins but closes only its event listener when the session ends. The next session opens another watcher. After two reloads, one file change produces several events and old instances can still read a folder that the new session should not use.

The acceptance test uses a temporary synthetic folder. It records watcher instance IDs, session IDs, open-handle count, and event count. Open one session, change one file, end the session, then start another and change one file. Each live session should receive only its own expected event; after shutdown and disablement, no instance should receive a new event. Inject failure immediately after watcher creation to prove the partial resource closes. If a callback was already processing a change at shutdown, test its cancellation or drain rule and ensure it does not publish into a later session.

Complete the plugin release acceptance record

Use the workbench below to list capabilities, resources, event semantics, and test evidence. Keep only synthetic IDs and controlled evidence locations in its browser-saved draft. A reviewer should be able to point to the exact residual capability, if any, and decide whether to block release.

Check	Pass evidence
Load and partial failure	Exact version and trusted source; registrations/resources expected; failed initialization leaves no live partial instance
Event contract	Per-event return and error behavior; approval failure blocks the write through an enforced path
Reload twice	One current callback per event; no stale tools, commands, permissions, watchers, or in-flight surprises
Disable	Old names cannot be called; no resource or authority remains; dependent write tools stay guarded
External effects	Completed and unknown writes have operation IDs and owners; teardown makes no claim to undo them

Common questions

Does agent_end mean the plugin can release every resource? No. In Pi, retries or queued work may continue after that event, and its docs direct session-scoped cleanup to session_shutdown. In the OpenAI Agents SDK, an agent-end callback is an agent-run hook, not a plugin unload event.

Will a disposer cancel a callback already running? Do not assume so. Pi explicitly says unsubscribing does not affect a dispatch already in progress. Set a stop flag or cancellation rule and test the host’s in-flight behavior.

Is a successful hot reload a rollback test? No. It tests replacement of registrations and resources. External writes require an independent operation ledger, lookup, and authorized correction path.

Boundary

This record covers executable plugin behavior in a named host and release. It does not prove that untrusted code is sandboxed, that an event hook enforces authorization by itself, or that every framework has Pi’s error semantics or Cordis’s effect handling. API versions and embedding hosts differ. Repeat the tests when a plugin, host, event contract, permission set, or dependency changes, and keep irreversible external effects under their own approval and recovery controls.