An AI agent harness is the software layer around an AI model that manages how an agent actually performs work. It can control the task lifecycle, tool use, context, state, execution environment, permissions, approvals, recovery, tracing, verification, and operational limits. The exact boundary depends on the architecture; not every harness needs every capability.
That distinction becomes important when an AI system stops being only a question-and-answer interface. A model can generate a useful answer in one turn. An agent may need to look things up, call tools (functions that do real things, such as searching a database or creating a record), react to the results, ask a person for approval, retry a failed operation, save its progress, and prove that the requested task was actually completed.
The engineering question therefore changes from “What can the model generate?” to “What does the surrounding system permit the model-driven agent to do?”
The model supplies intelligence. The harness supplies controlled execution around that intelligence.
| USER Task |
HARNESS Control |
MODEL Decision |
TOOLS Action |
SYSTEM Real effect |
Read left to right: a person gives a task, the harness controls the run, the model decides the next step, tools carry it out, and the real system changes. The harness sits between the model and everything it can affect.
📌 Simple analogy: Imagine giving a very smart worker a powerful toolbox. The worker can decide how to solve the problem, but the workshop still decides which tools are available, which machines are locked, when another person must approve an action, where the work happens, and how you know the job is really finished.
- What an agent harness actually is
- Model, agent, harness, application, tool, state, and sandbox
- What happens during an agent run
- When you need a harness—and when you do not
- Tools, permissions, and the execution boundary
- Context, state, memory, and recovery
- Human approval, completion checks, and safe stopping
- Tracing, evaluation, budgets, and operational limits
- Hands-on lab: build a tiny harness in 15 minutes
- Enterprise rollout
- Common mistakes
- Plain-English glossary
- FAQ
01
What an Agent Harness Actually Is
📌 Think of the harness as the workshop around the intelligence. The model may suggest what should happen next. The harness can decide how that suggestion becomes a controlled operation.
There is no single formal industry standard that assigns one mandatory set of components to an “agent harness.” Different sources draw the boundary in different places, and that is fine as long as you know which one you mean:
- Narrow view. Anthropic’s write-up on trustworthy agents lists four components of an agent: the model, a harness, tools, and an environment. There, the harness means the instructions and guardrails the model operates under, while tools and the environment are separate layers.
- Runtime view. Microsoft’s Agent Framework documentation describes a harness as the runtime wrapped around the model: it runs the loop that calls the model and executes requested tools, manages history and context, applies approval and safety policies, and keeps the agent moving toward completion.
- Engineering-environment view. OpenAI uses the phrase harness engineering for the broader environment, tooling, structure, and feedback systems that let agents do reliable work.
This article uses the runtime view, because it is the most useful for builders. A practical working definition:
An agent harness is the runtime and control layer that manages an agent run and its interaction with models, tools, context, state, execution environments, people, and external services.
The important word is manages. A harness does not need to own every piece of the application. It may be a small loop in one application, a reusable runtime, a framework abstraction, or a larger platform that includes durable state, sandbox sessions, approvals, tracing, and recovery.
The key distinction is between what the model proposes and what the system permits.
Engineering principle
Model output is not automatically execution authority.
OpenAI’s Agents SDK provides one documented implementation of this idea: its Runner can manage agent turns, tool execution, handoffs, guardrails, sessions, tracing, and resumable execution. The same documentation also explains when an application may choose a lower-level approach and own more of the loop itself.
That is why the word harness is useful. It focuses attention on the runtime around the model rather than pretending the model is the entire agent system.
Your system needs to answer not only “What did the model say?” but also “What was allowed to happen because of that output?”
02
Model, Agent, Harness, Application, Tool, State, and Sandbox
These terms are often mixed together because the boundaries can be implemented in the same process. Conceptually, however, separating them makes the system easier to design and secure.
| Concept | Think of it as | Main responsibility |
|---|---|---|
| Model | Decision engine | Produces outputs from the context it receives. |
| Agent | Task performer | Uses the model and available capabilities to pursue a task. |
| Harness | Control system | Manages execution around the agent, including whatever tools, state, policies, recovery, and observability the architecture requires. |
| Application | Product | Owns the user experience, domain services, identity, business rules, and surrounding infrastructure. |
| Tool | Capability | Performs an operation outside the model. |
| State | Work record | Preserves progress, status, pending decisions, identifiers, or other information needed beyond one model turn. |
| Sandbox | Isolated workspace | Provides a bounded environment when the agent needs files, commands, code execution, or similar capabilities. |
The model transforms supplied context into an output. That output can be text, structured data, a proposed tool call, a classification, or another supported result.
The agent is the model-based task performer. Current agent platforms commonly combine a model with instructions, tools, and runtime behavior such as handoffs, guardrails, sessions, or structured outputs.
The harness surrounds that execution. It can decide what enters the next model turn, which tools are exposed, whether a tool call may execute, how state is updated, when approval is required, and when the run must stop.
The application is larger than the harness in many designs. It may own the user interface, identity system, databases, event streams, business services, and domain-specific policies.
The tool is where an agent crosses from reasoning about an action into requesting an external operation. Google’s function-calling documentation makes this boundary explicit: the model can propose a function call, but application code executes the function and sends the result back.
The state is what the system needs to remember beyond the current turn. Microsoft’s Agent Framework documentation, for example, defines workflow state with explicit scope and visibility rules rather than treating state as one undifferentiated conversation transcript.
The sandbox is a possible execution boundary. OpenAI’s sandbox-agent documentation distinguishes persistent workspaces and sandbox sessions from the higher-level agent run. That is a useful reminder that “agent runtime” and “isolated execution environment” are related but not identical concepts.
Risk: The model, runtime identity, and execution environment become blurred together.
Control: Separate proposal, policy, authorization, and execution boundaries.
Remaining risk: A misconfigured downstream service can still perform a harmful operation even when the surrounding architecture looks disciplined.
You are designing an agent architecture and need to decide which responsibility belongs in the model, harness, application, tool, or isolated execution layer.
03
What Happens During an Agent Run
A useful way to understand a harness is to follow one task from start to finish.
📌 Simple analogy: Think of a delivery robot. Someone gives it a destination. The robot checks what it knows, chooses its next movement, observes the result, adjusts if necessary, and eventually stops. The control system decides how much movement is allowed and what should happen when something goes wrong.
receive task
↓
build allowed context
↓
ask model for next decision
↓
inspect proposed next step
↓
validate → authorize → execute
↓
record result and update state
↓
ask model again or stop
↓
verify completion
↓
return result
The important point is that one model response does not necessarily equal one completed task.
The model may propose a search. The harness may execute it and return the result. The model may then decide that another operation is necessary. A consequential action may require approval. A temporary failure may trigger a retry. A permanent failure may force the run to stop.
OpenAI’s Agents SDK describes this as an agent loop in which a run can continue through model turns, tool calls, handoffs, and other runtime events until it reaches a terminal outcome. Microsoft’s harness documentation similarly describes a tool-calling loop with a configurable iteration limit.
This is why the harness needs explicit boundaries. A run should always end in a clearly named outcome, for example: succeeded and verified, failed, cancelled, waiting for a human, or stopped by a limit.
Think like a runtime engineer
Ask what happens when the model is wrong, a tool is unavailable, a user changes their mind, an action partially succeeds, or the run continues longer than expected.
Useful limits can include maximum turns, timeouts, bounded retries, cancellation, concurrency limits, and resource budgets. The exact mechanism depends on the runtime and provider. The principle is independent of implementation: an agent should not be allowed to continue indefinitely merely because the model continues producing next steps.
Risk: An unconstrained loop can consume resources, repeat side effects, or continue after its assumptions have become invalid.
Control: Turn limits, timeouts, cancellation, retry policies, and idempotency-aware execution (safe-to-repeat operations, explained in section 06).
Remaining risk: Limits reduce runaway behavior but do not prove that a successful run produced a correct result.
Your agent performs more than one meaningful step, especially when those steps can affect external systems.
04
When You Need a Harness—and When You Do Not
Not every AI feature needs a sophisticated agent runtime. The right question is whether the task requires controlled multi-step execution.
📌 Think of complexity as a spectrum. You do not build an aircraft control room for a bicycle ride. The more autonomy, state, tools, side effects, duration, and operational risk you introduce, the more valuable a dedicated harness becomes.
In the table below, “harness depth” means how much runtime control you need to build or adopt: thin is a little application code around a model call, and strong is a full runtime with policy, state, and recovery.
| Scenario | Likely architecture | Harness depth | Reason |
|---|---|---|---|
| Summarize a document | Application → model | Thin | No autonomous loop or external side effect may be necessary. |
| Extract structured fields | Application → model → validation | Thin / moderate | Schema checks and recovery may matter, but autonomy can remain low. |
| Search through a tool | Model + tool loop | Moderate | Tool dispatch, failures, and runtime limits become important. |
| Create or update records | Agent + controlled tools + policy | Strong | External side effects need authorization, validation, and verification. |
| Long-running multi-step work | Agent + durable runtime/state | Strong | Recovery, checkpoints, cancellation, and resumability become relevant. |
| Run code or manipulate files | Agent + execution tools + possible sandbox | Moderate / strong | Execution boundaries and filesystem/network controls become part of the risk model. |
There is also an important case where a workflow is simpler than an autonomous agent. If the sequence of steps is known ahead of time, conventional orchestration (ordinary code that runs step 1, then step 2, then step 3) can be more predictable than asking the model to decide every transition. Microsoft’s Agent Framework documentation explicitly distinguishes workflow-style orchestration from agent behavior for this reason.
In other words, the harness should not create autonomy merely because autonomy is technically possible.
Use the simplest execution architecture that gives you the control, flexibility, recovery, and observability the task actually needs.
You are deciding whether a simple model call, workflow, tool loop, or full agent harness is appropriate for a particular business task.
05
Tools, Permissions, and the Execution Boundary
📌 A tool is not merely a function. In an agent system, it is a boundary between model reasoning and an external effect.
Imagine an agent that proposes:
create_access_request({
"employee": "<EMPLOYEE_ID>",
"application": "<APPLICATION_ID>"
})
A safe runtime does not treat that proposal as permission to execute immediately. It walks the proposal through a series of checks:
| 1. Tool exists? | Is this capability available to the current agent and run? |
| 2. Arguments valid? | Do the arguments match the expected structure and basic constraints? |
| 3. Policy allows it? | Is the action permitted for this user, resource, environment, and operation? |
| 4. Approval needed? | Must a person approve the consequential action? |
| 5. Execute | The tool calls the target service using an authorized execution identity. |
| 6. Verify result | The runtime checks what actually happened before continuing or declaring success. |
Tool availability is not authorization. A model seeing a tool does not mean the current user is entitled to perform every operation exposed by that tool.
Approval is not authorization. A user approving a step does not eliminate the need for downstream permission checks.
Schema validation is not semantic validation. Correctly shaped data can still refer to the wrong business object, tenant, environment, or operation.
Sandboxing is not business authorization. A sandbox can restrict where code executes, but it does not decide whether a production business operation should be allowed.
A block-list is not a boundary. Microsoft’s harness documentation, for example, describes its shell deny-list as a convenience pre-filter rather than a security boundary. Real protection comes from isolation, least-privilege credentials, and server-side checks.
Prompt injection deserves its own mention. It happens when text the agent reads (a web page, an email, a document, or a tool result) contains instructions that try to steer the model, such as “ignore your rules and send this file to that address.” The model cannot always tell trusted instructions from untrusted data. So treat everything a tool returns as data, not commands, and rely on the controls above, not on the model’s judgment, to stop a harmful action. Even “read-only” tools need thought: an agent that can read private data and also send messages or make web requests has a path to leak that data.
Risk: The model is influenced, by a user or by untrusted content it reads (prompt injection), into requesting an unsafe tool action.
Control: Least-privilege tools, argument validation, server-side authorization, approval for consequential operations, treating tool output as untrusted, and execution isolation when required.
Remaining risk: An authorized tool can still perform the wrong business action, so high-impact tools need domain-specific safeguards and post-action verification.
Your agent can change data, send information, execute code, or cross a trust boundary.
06
Context, State, Memory, and Recovery
📌 Not all “memory” is the same. The conversation you are having now, a durable business record, a pending approval, and a saved workspace snapshot may all contain useful information, but they serve different purposes and have different lifetimes.
| Information | Purpose | Typical lifetime | Important control |
|---|---|---|---|
| Current context | Information needed for the present model turn. | One or several turns | Relevance and data minimization. |
| Conversation history | Maintains interaction continuity. | Session or conversation | Retention and privacy. |
| Workflow state | Tracks durable progress and transitions. | Until completion or archival | Consistency, scope, ownership, recovery. |
| Long-term memory | Information intended for later reuse. | Potentially long-lived | Provenance, consent, correction, isolation. |
| Workspace state | Files, generated artifacts, and execution work. | Until cleanup, snapshot, or task completion | Isolation, permissions, cleanup. |
OpenAI distinguishes sessions from sandbox sessions and exposes mechanisms for preserving and resuming work. Microsoft documents explicit workflow state and state scopes. These implementations differ, but they point to the same lesson: state has scope, ownership, and lifetime.
A reliable harness should be able to answer four basic questions:
- What state exists?
- Who owns it?
- How long does it live?
- What happens if the process fails halfway through?
The last question is where recovery engineering begins. A checkpoint is a saved record of progress that a restart can resume from.
Imagine a fictional access-request agent that successfully creates a request but crashes before saving the request identifier it received. A blind restart would create a duplicate. The fix is to make the operation idempotent, meaning that doing it twice has the same effect as doing it once, and to save a checkpoint after each important step.
The harness sends the create operation together with an idempotency key, a unique label for this exact operation (for example, the run ID plus the employee and application). The service returns REQ-10482, and the harness saves it before allowing the next model step. If the process crashes after saving, recovery resumes from the saved state. If it crashes before saving, the restart sends the same key again, and a service that supports idempotency returns the existing request instead of creating a second. If the service does not support idempotency keys, the harness must first search for an existing request before retrying.
This is also where context engineering connects to harness engineering without becoming the same subject. Context engineering focuses on selecting and organizing what the model sees. Harness engineering also has to decide how that information is retrieved, scoped, persisted, refreshed, and protected during execution.
Risk: Stale, cross-user, poisoned, or over-broad state influences a later action.
Control: Explicit scope, tenant/user isolation, provenance, validation, retention rules, and recovery checkpoints.
Remaining risk: Stored information can still be wrong, so high-impact decisions should not blindly trust historical memory.
Your task can outlive one process, one session, or one model turn.
07
Human Approval, Completion Checks, and Safe Stopping
One of the most overlooked responsibilities of a harness is deciding when an agent is actually allowed to finish.
📌 Simple analogy: A student can say, “I finished the experiment.” The teacher may still inspect the output. Saying “done” and proving “done” are two different events.
A model-generated statement such as “The request was submitted successfully” is not automatically proof that the request was submitted successfully.
A stronger completion condition might look like this. A postcondition is a fact you can check in the real system after the action, such as “a request with this ID now exists.”
prepare
↓
approval
↓
execute
↓
receive external result
↓
verify postcondition
↓
mark run successful
That creates a useful distinction between model completion and system completion.
| Question | Model-centric answer | Harness-centric answer |
|---|---|---|
| What happens next? | The model proposes a next action. | Runtime policy decides whether that action can occur. |
| Can this action proceed? | The model may recommend proceeding. | Authorization, policy, approval, and tool controls determine this. |
| Are we finished? | The model may produce a final answer. | The system checks a defined completion condition. |
Human approval can be useful, especially for high-impact actions. But approval should be meaningful. A user repeatedly clicking “approve” on routine low-risk operations may not be providing useful oversight.
A practical design is therefore risk-based: low-consequence read operations may need no approval, selected write operations may need approval, and high-impact actions may require stronger gates or human review. Where approval is required, show the person a plain summary of exactly what will happen, and re-check permissions and state at execution time, because they may have changed while the request waited.
Risk: A user approves an action they did not fully understand, or the agent proceeds without meaningful confirmation.
Control: Risk-based approval, clear action summaries, least privilege, and revalidation before execution.
Remaining risk: Humans can still approve incorrect actions, and automated policies can still be incomplete.
“Done” has real business meaning. Define what must be true before your system declares success.
08
Tracing, Evaluation, Budgets, and Operational Limits
📌 Final answers are not enough. An agent can eventually produce a correct sentence after an incorrect tool sequence, excessive retries, or a surprising amount of unnecessary work.
A harness therefore needs visibility into the execution path, not just the final message. A trace is the step-by-step record of a run. Useful things to record include:
- Model turns and tool calls.
- Tool arguments and tool results.
- Failures and retry behavior.
- Approval requests and decisions.
- State transitions.
- Termination reason.
- Completion verification.
- Relevant resource or cost measurements.
OpenAI’s Agents SDK includes tracing for agent runs and exposes configuration related to trace identifiers and whether sensitive model or tool data is included. Microsoft’s harness lists built-in OpenTelemetry support. These are two implementations of one principle: an agent run should produce enough evidence to understand how it reached its outcome.
Budgets and limits are the other half of operating an agent safely and affordably. Because the context is typically re-sent to the model on every turn, cost can grow faster than the number of turns, so measure cost per run, not only per model call.
| Limit | What it prevents | Question to ask |
|---|---|---|
| Turn limit | Endless loops of model–tool cycles. | What is the maximum number of steps for this task type? |
| Time limit | Runs that hang or drag on. | When should the run time out and report failure? |
| Token or spend budget | Surprise bills from long contexts or retries. | What is the most one run may cost? |
| Tool-call budget | Repeated side effects, such as sending ten emails. | How many times may each write tool run per task? |
| Retry limit | Hammering a failing service. | Which errors are worth retrying, and how often? |
| Concurrency limit | Too many runs overwhelming a downstream system. | How many runs may act on this system at once? |
Harness evaluation is also different from a model benchmark. The harness itself should be tested.
| Test area | Question to verify |
|---|---|
| Tool correctness | Does the runtime validate arguments and handle tool failures predictably? |
| Policy enforcement | Does an unauthorized operation stop before the side effect occurs? |
| Recovery | Does the run resume from the right checkpoint without replaying unsafe operations? |
| Termination | Does the runtime stop cleanly on success, failure, cancellation, or budget exhaustion? |
| Completion verification | Does “success” correspond to a real postcondition? |
| Data isolation | Can state or tool results cross user, tenant, or workload boundaries? |
Instead of testing only whether an access-request agent produces a polite confirmation message, a harness test can verify that an unauthorized user cannot call the creation tool, that a retry does not create a duplicate, and that the run is not marked successful until a real request identifier is verified.
Risk: Execution failures are hidden by a seemingly good final response.
Control: Structured traces, useful run identifiers, failure categories, and carefully designed observability.
Remaining risk: Traces can contain sensitive information, so logging itself needs access control, retention rules, and data minimization.
Your agent is entering production or becoming important enough that someone will eventually need to investigate a failure.
09
Hands-On Lab: Build a Tiny Harness in 15 Minutes
Reading about a harness is useful, but running one makes it click. In this lab you will build a miniature harness in about 80 lines of Python. It uses a pretend model (a fixed script), so you need no account, no API key, no extra libraries, and it costs nothing. Nothing real can be changed: the “service” is just a Python dictionary.
A model that proposes steps, and a harness that checks each one, asks for approval, blocks unsafe requests, stops runaway loops, and refuses to call the run a success unless the result is verified.
python3 --version (on some systems the command is python --version). Create an empty folder and open a new file in it named tiny_harness.py.tiny_harness.py and save. Read the numbered comments first: (1) a fake service, (2) the list of tools the agent may ask for, (3) the pretend model, (4) the harness checks, (5) the final verification.import json
import uuid
# ---- 1. A pretend downstream service (stands in for a real system) ----
REQUESTS = {} # idempotency_key -> saved request
def lookup_employee(employee_id):
people = {"E100": "Asha", "E200": "Ravi"}
return {"employee_id": employee_id, "name": people.get(employee_id, "unknown")}
def create_access_request(employee_id, app, idempotency_key):
if idempotency_key in REQUESTS: # same key = same operation, so do not repeat it
return REQUESTS[idempotency_key]
record = {"request_id": "REQ-%05d" % (len(REQUESTS) + 1),
"employee_id": employee_id, "app": app}
REQUESTS[idempotency_key] = record
return record
# ---- 2. The tool registry: what the agent may ask for ----
TOOLS = {
"lookup_employee": {"fn": lookup_employee, "args": {"employee_id"}, "approval": False},
"create_access_request": {"fn": create_access_request,
"args": {"employee_id", "app"}, "approval": True},
}
ALLOWED_APPS = {"wiki", "payroll"} # simple policy; real systems check the user's identity
# ---- 3. A pretend model: it only PROPOSES the next step ----
def fake_model(history):
steps_done = sum(1 for m in history if m["role"] == "tool")
script = [
{"tool": "lookup_employee", "args": {"employee_id": "E100"}},
{"tool": "create_access_request", "args": {"employee_id": "E100", "app": "wiki"}},
{"final": "All done. Your access request was submitted."},
]
return script[min(steps_done, len(script) - 1)]
# ---- 4. The harness: validate -> authorize -> approve -> execute -> record ----
def refuse(trace, name, reason):
trace.append("DENIED %s: %s" % (name, reason))
return {"error": reason}
def execute(name, args, run_id, trace):
tool = TOOLS.get(name)
if tool is None:
return refuse(trace, name, "unknown tool")
if set(args) != tool["args"]:
return refuse(trace, name, "bad arguments")
if "app" in args and args["app"] not in ALLOWED_APPS:
return refuse(trace, name, "policy: app not allowed")
if tool["approval"]:
answer = input("Approve %s %s ? [y/N] " % (name, args))
if answer.strip().lower() != "y":
return refuse(trace, name, "human declined")
call_args = dict(args)
if name == "create_access_request":
call_args["idempotency_key"] = run_id + json.dumps(args, sort_keys=True)
result = tool["fn"](**call_args)
trace.append("EXECUTED %s %s -> %s" % (name, args, result))
return result
def run(task, max_turns=6):
run_id = uuid.uuid4().hex[:8]
history = [{"role": "user", "content": task}]
trace, created, final = [], None, None
for _ in range(max_turns):
proposal = fake_model(history)
if "final" in proposal:
final = proposal["final"]
trace.append("MODEL SAYS %s" % final)
break
result = execute(proposal["tool"], proposal["args"], run_id, trace)
if "request_id" in result:
created = result
history.append({"role": "tool", "content": json.dumps(result)})
print("\n".join(trace))
if final is None:
return "STOPPED: turn limit reached"
# ---- 5. Verify the real outcome, not the model's claim ----
if created and created in REQUESTS.values():
return "SUCCESS (verified %s)" % created["request_id"]
return "FAILED: the model said it was done, but no request exists"
print("RESULT:", run("Give employee E100 access to the wiki"))
python3 tiny_harness.py. When asked Approve create_access_request ... [y/N], type y and press Enter.✅ Expect to see: two lines starting with
EXECUTED, then MODEL SAYS All done..., then RESULT: SUCCESS (verified REQ-00001).n (or just press Enter).✅ Expect to see:
DENIED create_access_request: human declined, yet the model still says “All done.” The last line should be RESULT: FAILED: the model said it was done, but no request exists. This is the lesson of section 07: the model’s claim and the real outcome are different things, and the harness caught it."app": "wiki" inside fake_model to "app": "finance" and run again.✅ Expect to see:
DENIED create_access_request: policy: app not allowed without an approval prompt, because policy runs before approval. Change it back afterwards.fake_model, replace the whole create_access_request line with {"tool": "delete_everything", "args": {}} and run.✅ Expect to see:
DENIED delete_everything: unknown tool. The model asked for something that was never offered, so nothing ran. Restore the original line."final": "All done... from the script, then run and type y each time you are asked.✅ Expect to see: the same request repeated several times, always returning
REQ-00001 (that is the idempotency key preventing duplicates), and finally RESULT: STOPPED: turn limit reached. Put the line back.From toy to production. Every part of this miniature maps to something you would build for real:
| In the lab | In a production harness |
|---|---|
fake_model returns a fixed script |
A real model API call; its output is still only a proposal. |
REQUESTS dictionary |
A real downstream service that supports idempotency keys and does its own authorization. |
ALLOWED_APPS check |
Server-side policy based on the real user’s identity, tenant, and resource. |
input() approval prompt |
An approval UI with a clear action summary, an audit record, and revalidation at execution time. |
trace list of strings |
Structured traces with run IDs, access control, and retention rules. |
max_turns=6 |
Turn, time, tool-call, retry, and spend budgets, plus cancellation and an operator kill path. |
Final created in REQUESTS check |
Postcondition verification against the real system before marking the run successful. |
You are new to agents and want to see the loop, the checks, and the verification working end to end before you design anything larger.
10
Enterprise Rollout
A prototype asks whether the agent can perform the task. An enterprise deployment must also answer who owns it, how it is changed, what it records, how it is stopped, and how an incident is investigated.
| Concern | What the harness or surrounding platform should address | Operational question |
|---|---|---|
| Identity | Associate execution with the correct user, service, tenant, or workload identity. | Whose authority is represented at execution time? |
| Secrets | Keep credentials out of model-visible context and use controlled retrieval. | Could a prompt, tool output, trace, or error message expose credentials? |
| Change control | Version tools, policy logic, configuration, and agent behavior. | Can you identify what configuration produced an action? |
| Observability | Record enough evidence to diagnose success and failure. | Can an investigator reconstruct the run? |
| Retention | Define what prompts, tool results, state, traces, and artifacts remain stored. | What should be deleted, restricted, or minimized? |
| Incident response | Provide disablement, cancellation, credential revocation, and investigation paths. | How quickly can unsafe behavior be stopped? |
Identity deserves particular care as agents gain access to more systems.
The Model Context Protocol (MCP) authorization specification gives a concrete example at the protocol boundary. For HTTP-based transports it covers concerns such as validating that a token was issued for the receiving server (audience validation), careful token handling, secure transport, and authorization-code protection with PKCE. Two cautions: authorization is optional in MCP, and the specification is revised over time, so check the newest revision. These are protocol-specific controls, not universal rules for every harness.
An enterprise harness should also have a kill path: a way for operators to disable a tool, suspend an agent, revoke an identity, or stop new runs without depending on the model to decide that it should stop.
Risk: A production agent continues operating after a configuration issue, compromised credential, or unsafe behavior is detected.
Control: Centralized disablement, revocable credentials, bounded permissions, monitoring, and operator cancellation.
Remaining risk: Stopping future actions does not automatically undo actions that already happened.
An agent affects real users, production systems, sensitive data, money, or operational decisions.
11
Common Mistakes
Prompts are useful for instructions, but mandatory authorization, state transitions, limits, and other hard controls should live in runtime or infrastructure enforcement where practical.
A tool description tells the model what the capability is supposed to do. It does not replace downstream authorization and business-rule enforcement.
Broad credentials increase the blast radius (the amount of damage possible) when a model, tool, or configuration behaves incorrectly. Prefer narrow capabilities and narrowly scoped identities.
A transient network error and an authorization failure are not the same thing. Retries should be classified, bounded, and designed around the side effects of the operation.
Conversation history helps the agent maintain continuity. It should not automatically become the authoritative source for business state.
Sandboxing can reduce execution risk, but it does not automatically solve business authorization, identity, data governance, or downstream API policy.
Approval loses value when users approve routine actions without meaningful review. Match approval requirements to consequence and risk.
Natural-language confirmation is not the same as a verified external result. Define and check a real postcondition.
Insufficient logs make incidents difficult to investigate. Uncontrolled logs can expose sensitive information. Observability needs purpose, access control, and retention rules.
Define the run lifecycle, tool contract, permission boundaries, state model, completion criteria, and recovery requirements first. Then choose the smallest implementation that satisfies them.
Web pages, emails, documents, and tool results can contain instructions aimed at the model. Treat them as untrusted data and enforce permissions outside the model.
Start with the task. Identify where the model must decide. Identify what the outside world can change. Put mandatory rules around those changes. Add durable state when the task outlives one process. Add tracing so the execution can be inspected. Add a sandbox when the work genuinely requires execution isolation.
You are deciding whether your current agent architecture is actually engineered, or simply wrapped around a prompt and a few tool calls.
GLOSSARY
📖 Plain-English Glossary
| Term | What it means |
|---|---|
| Agent | An AI model that decides its own next steps and uses tools to get a task done. |
| Tool call | The model’s request for something outside itself to happen, such as a search or a database update. The model asks; the harness decides. |
| Context window | The amount of text a model can consider in one turn. Everything it needs to know must fit inside. |
| Guardrail | A rule or check that limits what the agent may do or say. |
| Least privilege | Giving a tool or identity only the access it needs and nothing more. |
| Blast radius | How much damage can happen if something goes wrong. |
| Prompt injection | Hidden or hostile instructions in content the agent reads, aimed at steering the model. |
| Idempotent / idempotency key | Safe to repeat: doing it twice has the same effect as once. The key is a unique label that lets a service recognize a repeat. |
| Checkpoint | A saved record of progress that a restart can resume from. |
| Postcondition | A fact you can check in the real system after an action to prove it worked. |
| Trace | The step-by-step record of an agent run, used for debugging and audits. |
| Sandbox | An isolated workspace that limits what code or commands can touch. |
| Kill path | A way for operators to stop an agent or revoke its access without relying on the model. |
FAQ
❓ Frequently Asked Questions
READ FURTHER
🔗 References & Further Reading
These official or primary sources (vendor documentation, standards bodies, and security projects) were used to check the architecture and security concepts discussed here.
- OpenAI — Harness engineering: leveraging Codex in an agent-first world
- OpenAI Agents SDK — Agents, runners, tools, sessions, tracing, and sandbox agents
- OpenAI Agents SDK — Running agents
- OpenAI Agents SDK — Sandbox agents
- Anthropic — Trustworthy agents in practice
- Anthropic Claude Code — Security
- Google AI — Function calling
- Microsoft Agent Framework — Agent Harnesses
- Microsoft Agent Framework — Workflow State
- Model Context Protocol — Authorization (2025-06-18 revision; newer revisions exist)
- Model Context Protocol — Specification index (choose the newest revision)
- NIST — AI Agent Standards Initiative
- OWASP GenAI Security Project — Top 10 for Agentic Applications
Source note: Vendor-specific behaviors are described as documented architecture examples, not universal rules. Documentation changes often, so confirm details against the current pages. Product and project names belong to their respective owners.
TAKEAWAY
📝 Summary
- An agent harness is the runtime around the agent.
- The model proposes; the runtime controls what happens next.
- Tools are execution boundaries and therefore need validation and authorization.
- Treat text the agent reads as untrusted data, because prompt injection is a real risk.
- Context, conversation history, durable state, memory, and workspace are not interchangeable.
- A sandbox can reduce execution risk but does not replace business authorization.
- Human approval should be meaningful and risk-based.
- System completion should be verified rather than inferred from the model’s final sentence.
- Limits, tracing, and recovery make agents operationally manageable and affordable.
- The best harness is the smallest one that reliably enforces the controls your task actually needs.
Final thought
An AI model can provide flexible reasoning, but the surrounding harness determines how that reasoning becomes an executable, observable, bounded, and recoverable system.
Comments
Post a Comment