Search the problem you are solving—not the buzzword.

Decide whether AI belongs in the task, choose who controls each next step, then define behavior you can test with real inputs and a human fallback.

Map instructions and sources, preserve decisions as conversations grow, define acceptable content and a consumable output format, then test untrusted content.

Turn opinions about AI quality into representative cases, acceptance checks, failure categories, and a release decision another reviewer can reproduce.

Decide when retrieval is justified, approve its source versions, test eligible passages and updates, then route insufficient evidence and verify answer claims.

Choose a delegation topology, contract tool effects, vet MCP servers and plugin lifecycles, guard the loop, keep memory correctable, and inspect or resume long runs.

Set the smallest useful authority for each agent action, put human review before consequential effects, and rehearse a safe recovery path.

Budget memory, test quantization and adapter training, choose an engine, measure one-instance capacity, plan GPU replicas and failure reserve, then compare with a cloud API.

Choose and operate models with task evidence, full workflow cost, measured cache usage, latency, baseline, privacy, and recovery constraints.

Four implementation teardowns—support triage, content review, research synthesis, and invoice approval—each with its evidence gate, rejected version, and recovery path.

Move from an attractive idea to a bounded task, a workflow-or-agent decision, and a ten-case pilot a colleague can review before further investment.

Preserve the right context, define content and format gates, test untrusted and difficult inputs, then diagnose failures in evidence, instructions, parsing, or handoff.

Choose an answer method, approve source versions, then test chunking, retrieval, access, index updates, citations, freshness, and no-answer behavior.

Build a bounded tool-using workflow with control ownership, data and action authority, stop rules, human review, recovery, service fallback, cost, and drift checks. Test delegation, MCP, memory, models, and deployment only when your task needs them.

Judge a task by variation, verifiability, stakes, data boundaries, and fallback before choosing a model or building a workflow.

Replace a vague AI feature request with a one-page brief that names the user, input, acceptable output, evidence, boundaries, and fallback.

Build a deliberately small test set with ordinary, difficult, ambiguous, and unsafe cases so a promising demo becomes evidence.

Separate instructions, source material, examples, history, tools, and output rules to see what the model actually receives.

Define required fields, evidence, tone, uncertainty, and rejection conditions before optimizing prompt wording.

Trace a bad answer to missing input, conflicting instructions, weak evidence, capability limits, or a broken handoff.

Compare knowledge volatility, source volume, citation needs, access rules, and maintenance cost before adding retrieval.

Audit whether each important claim is supported by the cited passage, uses the right version, and survives a freshness check.

Place each action on a five-level permission ladder, from read-only suggestions to explicitly approved irreversible work.

Write the stop signal, saved state, owner, rollback action, and safe retry rule before automation reaches production.

Weight quality, latency, tool use, privacy, recovery, and operating constraints using cases from the workflow you will ship.

Count input, output, retries, tools, waiting, review, and failure recovery instead of comparing a single token price.

Route every field through allow, transform, isolate, or exclude so a useful workflow does not quietly become an uncontrolled data transfer.

Sample real task slices, write reviewable references, calibrate graders, and version the set so model changes produce trustworthy comparisons.

Choose review gates from impact, reversibility, and uncertainty, then give reviewers the evidence and actions needed to make a real decision.

Watch task mix, inputs, dependencies, outcomes, and human overrides against a named baseline so every alert leads to an inspectable response.

Build a small retrieval trial that checks gold passages, no-answer questions, document versions, ranking, and access boundaries before generation.

Build a regression set for untrusted pages, files, retrieval snippets, and tool results; check intended behavior and permission gates before release.

Measure cache writes, reads, latency, and accepted-task cost with a stable prefix and a changed-prefix control, using provider-specific rules.

Decide when a fixed AI workflow is enough and when an agent needs bounded control, tool feedback, stop rules, and a human handoff.

Specify an agent tool's input schema, preconditions, authorization, side effects, idempotency, timeout results, and trace before making it callable.

Use observable progress, separate turn and cost budgets, error classes, and duplicate-action checks to govern each step of an agent run.

Set a compaction checkpoint from measured context use, preserve decisions and unresolved actions, then test whether the next turn still follows the facts.

Diagnose the failure first, then compare instructions, request-time retrieval, and fine-tuning on the same holdout cases and operating gates.

Compare fixed windows, overlap, and document structure on clauses and tables, then choose chunk boundaries using complete evidence, citations, and cost.

Estimate raw model weights and KV cache separately, test context length and concurrency, then measure the missing peak costs on your serving stack.

Compare one agent, a manager calling specialists, a conversation handoff, and parallel delegation by control ownership, shared state, cost, and failure recovery.

Design a cross-session memory rule for what to save, how to handle a changed preference, and how to verify deletion across retrieval and retained copies.

Compare an unquantized BF16 reference with a named 4-bit package on fixed task cases and serving load before trading quality for memory.

Compare owned hardware, a team-managed rented GPU, and a managed model API using data paths, measured peak capacity, operating ownership, and cost per accepted task.

Find the highest tested arrival rate that meets first-token, output-speed, queue, error, and memory targets on your actual inference server.

Carry verified identity, tenant scope, and document ACLs into every chunk, then test denial across retrieval, model context, citations, cache, and revocation.

Route a failed model request by error type, user deadline, data boundary, and an evaluated backup model instead of treating every outage as a retry.

Reconstruct a single agent run with spans, tool call IDs, approval changes, and external state to find the first evidenced divergence without exposing sensitive trace data.

Vet a third-party MCP server's discovered tools, OAuth or local credentials, data destinations, version changes, failure behavior, and revocation before an agent can use it.

Match an AI response format to its downstream consumer, then route refusals, truncation, parser errors, schema errors, and factual errors separately.

Compare BM25, dense vectors, hybrid fusion, and reranking on exact IDs, paraphrases, and mixed questions while holding evidence, access, context, and latency gates fixed.

Compare rule checks, an LLM judge, and independent human decisions on the same outputs; find false passes by task slice before trusting automated scores.

Design a durable checkpoint, approval and operation ledger, and single-owner resume gate for agent jobs that outlive one process or wait for a person.

Route calibrated binary probabilities into automatic approval, automatic rejection, and human review using error costs and reviewer capacity, then validate the rule on held-out cases.

Trace a source revision or withdrawal through ingestion, chunk and vector cleanup, search visibility, caches, and answer probes before calling a RAG knowledge base current.

Route a RAG request to a supported answer, a specific clarifying question, an honest unable-to-verify response, or human review when retrieved evidence is incomplete.

Decide which exact documents and revisions may ground a RAG answer by checking authority, ownership, use rights, audience, effective dates, and conflicts before indexing.

Compare parsed content with original pages and cells before chunking. Check OCR, reading order, table relationships, figures, and source locations.

Count the next complete request, reserve room for tool results and all generated tokens, then check model input, output, and context limits before sending it.

Run observable allow-and-deny probes for an agent execution environment's mounts, egress, commands, resource limits, and credentials before it handles untrusted work.

Decide whether a recurring task has a stable trigger, inputs, steps, acceptance test, and owner before packaging it as a reusable agent skill.

Shortlist a local LLM server by model package, hardware, API behavior, and operating needs, then run the same acceptance and service trial before committing.

Admit a stable behavior problem, split training data without leakage, probe real training memory, and compare adapters with a frozen baseline on unseen cases.

Test registrations, event errors, resource cleanup, two hot reloads, disablement, partial initialization, and external effects before shipping an agent extension.

Turn a tested per-replica arrival rate into a candidate replica count, then verify GPU placement, failure capacity, warm-up time, and rollout peaks.

Build a provider-aware round trip that assembles complete calls, validates before execution, pairs every result, and resumes from durable state.

Separate client, gateway, model, tool, and rendering delays in one request trace before changing a model or adding hardware.

Compare sampling controls on repeated, frozen cases; measure task errors and useful variation without assuming temperature zero guarantees identical answers.

A support team tested AI queue suggestions on ten awkward tickets, found the failure that mattered, and shipped a review gate instead of autonomous routing.

A content team let AI draft a policy update, then required every consequential claim to pass a source, version, and freshness check before publishing.

A research team used retrieval to draft a regulatory brief, then a source revision landed before the meeting and forced a claim-by-claim re-check.

An operations team built an agent to prepare invoice approvals, then removed its payment tool, bound approval to invoice scope, and rehearsed timeout and duplicate outcomes.

A production-readiness review for MCP servers covering per-user credentials, tool allowlists, tenant boundaries, rate limits, audit logs, and failure isolation.

A practical regression workflow that turns one production failure into a labeled case, a pass condition, and a gate that runs before the next prompt or model change.

A cost model that counts tokens, retries, tools, human review, and failure recovery per accepted outcome—not per API call.

A team workflow for coding agents that gates the task, the diff, and the runtime before a human merges—without turning review into a rubber stamp.

A practical trace schema for agent runs that records identity, tool effects, approvals, state changes, and costs—without storing secrets or full prompts.