Skip to main content

AI Agent Harness Engineering: State Management in an AI Agent Harness — Sessions, Context, and Resuming Work

Calculating read time…

State management in an AI agent harness is the engineering discipline of deciding what must survive a run, what the model should see on the next turn, what can be safely forgotten, and how an interrupted task can continue without repeating or corrupting work.

A useful harness does not simply “remember everything.” It keeps different kinds of state at different lifetimes, scopes them to the right user and task, reconstructs a bounded context for the model, and verifies recovery before allowing more tool actions.

Imagine an agent reviewing an invoice. It has already read the purchase order, compared three line items, called an internal validation tool, and is waiting for a human approval before sending a correction request. The process stops because the worker restarts. When the agent comes back, the important question is not merely, “Does it remember the chat?” The real question is: Can the harness reconstruct the right state, prove what already happened, identify what still needs to happen, and safely continue?

Engineering principle

Continuity is not the same thing as memory. A transcript can preserve conversation history while still being a poor recovery record. A durable state record should tell the harness what stage the task reached, which external effects were confirmed, and what may safely happen next.

📑 In This Post

AT A GLANCERun State, Session, Context, Durable State, and Checkpoint

Thing Main question Typical contents Primary job
Run state What is happening right now? Current step, pending tool call, transient data, interruption Drive the current execution safely
Session Which ongoing interaction is this? Session ID, conversation history, session metadata Connect multiple runs that belong together
Model context What should the model see now? Selected history, task facts, tool results, instructions, retrieved material Give the model the information needed for this decision
Durable business state What must remain true after the process ends? Workflow status, approvals, external IDs, artifact references Support reliable recovery and business correctness
Checkpoint Where can execution restart? Versioned snapshot, completed steps, pending work, verification markers Resume without guessing or duplicating effects

01What State Actually Means in an Agent Harness

🧒 Child-friendly analogy

Think about a board game. The conversation is what the players said. The game state is whose turn it is, which pieces have moved, which rules have been unlocked, and whether someone is waiting for a dice roll. You cannot reliably resume the game just by remembering the conversation.

In an agent harness, state is the set of information that the runtime needs to understand the current execution and make a correct next decision. That can include conversation history, workflow status, tool results, approval status, external identifiers, task checkpoints, and references to artifacts. The exact fields depend on the architecture.

The model is not the durable owner of that state. The model produces a proposed next action from the context it receives. The harness decides how that context is assembled, whether a tool may run, whether a prior action is already complete, and whether the run is allowed to continue.

🟢 Worked example (fictional scenario)

An internal invoice agent receives INV-1042 as task TASK-1842. After validation, its durable workflow record might say:

task_id: "TASK-1842"
invoice_id: "INV-1042"
schema_version: 3          # format of this record
revision: 8                # increases with every committed change
stage: "awaiting_approval"
purchase_order_checked: true
validation_result: "passed"
pending_effect: "send_correction_request"
correction_request: "not_sent"
approval_required: true

Notice what is missing: a giant copy of every earlier message. The workflow record captures the durable facts that matter for recovery, while the model context can be rebuilt from those facts plus selected history. Notice also that schema_version (what shape the record has) and revision (how many committed changes it has seen) are different things. Section 08 explains why that matters.

This leads to a useful rule: store facts and transitions explicitly; do not depend on the model to infer the entire workflow from a transcript.

🎯 Use this when...

A task can pause, span multiple worker runs, call tools, require approval, or create external side effects.

02Session vs Context vs Durable State

🧒 Child-friendly analogy

A session is the folder for one conversation. Context is the handful of pages you put on the desk for today’s question. Durable state is the official record that says what actually happened. They are related, but they are not interchangeable.

Session. A session groups related work. Some frameworks persist conversation history behind a session identifier. For example, the OpenAI Agents SDK (Python) documents session backends that load stored history before a run and save the new items afterward, and a run that paused for approval can be resumed against the same session. The same documentation says a session cannot be combined, in one run, with OpenAI’s server-managed continuation options, so you choose one mechanism or the other. See the OpenAI Agents SDK session documentation.

Context. Context is the actual material presented to the model for a specific inference step. A session may contain far more material than should be sent every time. A harness can retrieve recent history, durable task facts, selected tool results, and other relevant information, then construct a bounded context.

Durable state. Durable state is the authoritative record of facts that must survive worker restarts, context changes, and model changes. It should not depend on the wording of a previous model response. A business workflow database, task store, event log, object store, or other persistence layer can serve this purpose depending on the architecture.

Checkpoint. A checkpoint is a recoverable execution position. It tells the harness enough to decide what to do next. It may be represented inside a workflow system, a serialized run state, or a combination of database records and external workflow metadata.

Architecture note

There is no universal requirement that one product must provide all four layers. Some SDKs keep history on your side (a session you host), while some APIs can keep conversation state on the provider’s side. Durable-execution engines and graph frameworks offer their own checkpointing, under different names. These are different implementation boundaries, not a single industry standard, so map each layer to who owns it in your design.

🛡 Safety Check

Risk → The application treats a session ID as proof that the caller is allowed to view or modify that session.

Control → Authenticate the caller separately and authorize access to the session and its underlying storage. The OpenAI Agents SDK documentation states that a session ID only selects history; it does not authenticate a user or authorize access to that history, and it adds that the SQLite backend does not detect edits, deletions, or reordering of stored rows. See the storage trust-boundary notes.

Remaining risk → A correctly authorized caller can still receive stale, poisoned, over-retained, or cross-workflow state if the state model itself is poorly designed.

03Choose State by Lifetime, Scope, and Authority

🧒 Child-friendly analogy

Keep your lunch order, your school timetable, and your passport in different places. You would not carry the passport into every lunch conversation, and you would not trust a sticky note to be the official passport record.

A practical state model starts with three questions:

  1. How long must it survive? One tool call, one run, one session, or an entire business process?
  2. Who may see it? One invocation, one user, one tenant, a team, or an operational service?
  3. Who is authoritative? The model, the application database, an external system, or a human approval record?
State class Example Lifetime Design concern
Ephemeral run state Current tool result, pending interruption Current execution Serialization and safe replay
Session state Conversation history and session metadata Conversation or work thread Isolation, retrieval, retention
Workflow state stage, approval record, external reference Business process Authority and consistency
Artifact state Report, file, test output Until retention policy ends Integrity, ownership, expiry

A strong harness also records state transitions, not just final values. “Approval received” is stronger than a mutable boolean because the harness can attach a timestamp, actor identity, approval scope, and the exact action that the approval authorizes.

state_transition:
  revision: 8
  from: "validated"
  to: "awaiting_approval"
  actor: "harness"
  reason: "correction_request_requires_human_approval"

approval_record:
  revision: 9
  approval_id: "APR-5531"
  actor: "finance-reviewer-17"
  action_scope: "send_correction_request"
  payload_digest: "sha256:3b7c1e..."   # hash of the exact request body approved
  expires_at: "2026-10-05T12:00:00Z"

The exact schema will differ by system. The design idea is the important part: make authority-bearing facts explicit and versionable. Binding the approval to one action, one exact payload, and an expiry time means a later, different request cannot quietly reuse it.

🎯 Use this when...

You need to decide what belongs in a database, what belongs in a transcript, and what can disappear after the run.

04Rebuilding a Useful Context for the Model

🧒 Child-friendly analogy

Suppose you have a huge school library, but your teacher asks one question. You do not carry every book to the desk. You bring the few pages that help answer today’s question.

Context reconstruction is the harness deciding which stored information becomes the model’s next input. This is where context engineering touches harness engineering: the harness owns the retrieval and assembly mechanism, while the selected context determines what information is available to the model for the current decision.

A practical reconstruction pipeline can look like this:

  1. Authenticate the caller and authorize access to the session and workflow records.
  2. Load the latest checkpoint and check its schema version.
  3. Fetch only the history and artifacts relevant to the next step.
  4. Remove or summarize material that no longer earns its space.
  5. Attach current tool observations rather than replaying stale guesses.
  6. Apply the current permissions and policy context.
  7. Build the final bounded model input.
🟢 Worked example: rebuilding TASK-1842

The harness discovers that TASK-1842 is waiting for approval. It does not need the entire raw conversation. It can construct a compact context containing:

  • Current task objective: correct a validated invoice mismatch.
  • Confirmed facts: purchase order matched; validation passed.
  • Current state: correction request not yet sent.
  • Approval record: approval required and still pending.
  • Recent tool evidence: last validation result and timestamp.
  • Task instruction: continue the review and prepare the correction, but do not send it before approval.
context = {
  "goal": "resolve_invoice_mismatch",
  "stage": "awaiting_approval",
  "verified_facts": ["po_match", "validation_passed"],
  "pending_effect": "send_correction_request",
  "approval": "pending",
  "latest_tool_observation": {
    "tool": "validate_invoice",
    "result": "passed",
    "observed_at": "2026-10-04T09:12:00Z"
  }
}

This is intentionally illustrative pseudocode, not a vendor SDK. The important pattern is to separate authoritative facts from model-generated narration. Also note that the instruction “do not send before approval” helps the model plan, but it is not the control. The control is the harness refusing to run the send tool until it finds a valid approval record.

For long-running work, context reduction becomes important. OpenAI’s compaction guide describes server-side and standalone compaction. The compaction item it produces is opaque and not meant to be read by humans, and for the standalone endpoint the returned window is meant to be passed on as it is rather than pruned. See OpenAI’s compaction guidance. Anthropic’s prompting guidance for multi-window work takes a complementary approach: save progress and state outside the context window (for example a structured state file, free-form progress notes, and version-control history) so that a fresh window can pick the task up, and consider starting a fresh window instead of compacting when the state can be rediscovered from files. See Anthropic’s long-horizon state guidance.

Engineering principle

A summary is a derived view, not automatically the source of truth. When summarization changes what the model sees, keep critical workflow facts outside the summary and reconstruct them from authoritative state.

🛡 Safety Check

Risk → A stale or attacker-influenced summary becomes the next turn’s “truth.”

Control → Keep critical facts separately typed, versioned, and revalidated at recovery boundaries. Treat model-generated summaries and retrieved external text as untrusted inputs unless verified.

Remaining risk → A malicious or incorrect fact can still enter the durable state through a legitimate workflow path, so write operations and state transitions require their own validation and authorization.

05How Pause, Resume, Retry, and Recovery Really Work

🧒 Child-friendly analogy

A bookmark only tells you the page number. A good save file also records which quests are complete, which items you already collected, and whether you were standing before or after opening a locked door.

Safe resumption is a reconciliation problem. The harness must compare its checkpoint with the outside world before continuing. This matters because a crash can happen after a tool succeeded but before the harness recorded success.

Now suppose the reviewer has approved the correction (revision 9). Consider what happens next:

1. Harness verifies approval APR-5531 and records its intent first
   (revision 10: action, idempotency key, approval_id).
2. Harness asks the tool to create the correction request,
   passing the idempotency key.
3. The external system creates request CR-771.
4. The network connection drops. The harness never sees the reply.
5. The worker restarts.
6. The checkpoint (revision 10) says: send attempted, outcome unknown.
7. Harness looks up the key in the external system
   before it makes any new create call.

Step 1 is the important design move. If the harness had written nothing before calling the tool, then after the restart it would not even know a send had been attempted. Recording the intent first, with a stable idempotency key, gives recovery something to look up.

The safest recovery is not “ask the model what probably happened.” It is to query an authoritative status source, use an idempotency key when the external system supports it, or route the ambiguous case to a controlled recovery path.

🟢 Worked example: recovery decision
  1. Load the checkpoint at revision 10.
  2. Verify the session belongs to the authenticated user and the correct tenant.
  3. Look up the idempotency key TASK-1842:send_correction:v1 in the correction system.
  4. If CR-771 exists, record the confirmed external ID and continue from the next safe step.
  5. If nothing exists and the system deduplicates on that key, retry within the retry budget using the same key.
  6. If the outcome cannot be established, or the system cannot deduplicate, stop and surface the ambiguity for review.

OpenAI’s Agents SDK reference for RunState shows how one framework handles a related problem. A paused run can be serialized (including its approval state) and resumed later. The documentation says to resume against the original session backend and session ID, with exclusive access to that history, and that the runner reconciles a pending history write without re-running the tool. It also warns that independently restored copies of a snapshot must not be resumed at the same time against the same session. See Run State.

🛡 Safety Check

Risk → A serialized run snapshot contains approvals and pending tool calls. If the snapshot comes from the client, or from storage anyone can edit, a caller could forge an approval.

Control → Keep snapshots on the server in trusted storage, verify their integrity and ownership before loading them, and apply approval decisions from authorized reviewers to that server-owned state. The same SDK reference advises against passing client-supplied state straight into deserialization and notes that deserialization does not authenticate the snapshot.

Remaining risk → A trusted store can still be misconfigured, backed up insecurely, or written by a bug. Integrity checks reduce this risk; they do not remove it.

A recovery routine should therefore answer four questions before allowing another consequential tool call:

  1. Identity: Is this the correct user, session, tenant, and workflow?
  2. Position: What is the last verified checkpoint?
  3. Effects: Which external actions are confirmed, unknown, or incomplete?
  4. Authority: Does the current run still have permission to perform the next action?
🎯 Use this when...

Your agent can be interrupted during tool calls, approvals, deployments, payments, record updates, or any other externally visible operation.

06Security: State Is Part of the Attack Surface

🧒 Child-friendly analogy

Imagine a notebook that your robot reads every morning. If anyone can quietly write “the teacher already approved this,” the robot may act on a lie the next day. A notebook that influences future actions is part of the control system, not ordinary scratch paper.

Persistent memory and state can increase usefulness while creating another path for bad or incorrect information to influence later runs. The OWASP Top 10 for Agentic Applications includes a dedicated entry on memory and context poisoning (ASI06), and a May 2026 post from the OWASP GenAI Security Project argues that memory should be treated as part of the attack surface, using a disclosed Claude Code persistent-memory issue as its example. See OWASP’s memory and context poisoning discussion.

For harness engineers, that means state should be protected with the same seriousness used for other control-plane data. The main questions are:

  • Scope: Can one user or tenant read another session?
  • Integrity: Can stored state be modified without authorization?
  • Provenance: Do you know whether a fact came from a user, a trusted system, a tool result, or model inference?
  • Retention: Are you keeping sensitive state longer than the workflow needs it?
  • Secrets: Are credentials, tokens, or raw personal data being persisted into conversation history or summaries?
🛡 Safety Check

Risk → Persistent state carries an instruction-like value from an untrusted document or earlier tool result into a later run.

Control → Separate data from authority: store provenance, validate state-changing fields, constrain which state can affect permissions, and require policy checks at the point of consequential action. Never treat persisted text as an authority grant.

Remaining risk → Context poisoning can still shape model reasoning, and legitimate state may become stale. No single filter, prompt, or storage mechanism eliminates this class of risk.

A particularly important boundary is the difference between state that helps the model reason and state that grants authority. A note saying “customer approved the change” should not itself be sufficient to authorize an API call. The approval record should be verified by the harness from an authoritative source.

The same principle applies to secrets. Keep credentials in a dedicated secret mechanism rather than conversation history or general-purpose agent memory. When the agent needs a credential, the harness should mediate access rather than teaching the model the secret.

Retention is another engineering decision. Some state can expire quickly; other records may be required for audit or business continuity. The right retention period is determined by the application, legal requirements, and risk model, not by the convenience of the agent.

07Testing State Management and Recovery

🧒 Child-friendly analogy

Do not test only whether the robot can finish when nothing goes wrong. Turn the lights off halfway through the game. Then ask whether it can reopen the same game without moving the pieces twice.

Harness evaluation should include both happy-path continuity and failure-path recovery. A session feature can appear to work while failing exactly when a production system needs it most.

Minimum state-management test matrix:

Test Injected condition Expected property
Resume after restart Terminate worker after checkpoint Resume from durable state without losing verified progress
Unknown tool outcome Drop response after external write Reconcile before retrying; no unintended duplicate effect
Session isolation Attempt to load another user’s session Authorization failure before context assembly
State schema migration Load an older checkpoint Deterministic upgrade or controlled rejection
Context compaction Replace large history with compact state Critical task facts and recovery behavior remain correct
Concurrent resume Two workers restore the same task at once Only one proceeds; the other is rejected or waits (for example by a stale-revision check)
Stale or mismatched approval Expired approval, or a payload that differs from the approved one Tool call blocked; new approval requested

For every recovery test, assert more than the final text. Inspect the state transition, tool-call count, external side effects, authorization decision, and trace. A fluent answer can hide a duplicate API call.

🛡 Safety Check

Risk → Evaluation checks only that the agent eventually produced a correct-looking response.

Control → Test side effects, state transitions, permissions, recovery, and trace evidence, not only natural-language output.

Remaining risk → Test suites cover known failure classes. Production can still encounter novel races, provider outages, malformed state, or unexpected tool behavior.

🎯 Use this when...

Your team is preparing an agent for long-running tasks, multi-worker execution, human approval, or business-critical tool calls.

08Enterprise Rollout

For an enterprise harness, state management becomes an ownership problem as much as a coding problem. Decide who owns the session lifecycle, workflow records, retention policy, recovery procedures, and approval boundaries.

A practical rollout sequence:

  1. Define the state contract. Document fields, provenance, allowed transitions, versions, and retention.
  2. Pick the persistence boundary. Decide which state belongs in session storage, workflow storage, artifact storage, or external systems of record.
  3. Instrument recovery. Log checkpoint creation, restoration, state-version changes, tool reconciliation, approvals, and recovery failures.
  4. Gate consequential actions. Enforce authorization and policy at tool execution time, even when earlier context suggests approval.
  5. Exercise incidents. Practice corrupted state, unavailable stores, stale checkpoints, cross-tenant access attempts, and partial external effects.

Version your state in two separate ways. The revision counts committed changes to one task and lets you detect a stale writer (two workers racing to update revision 10). The schema_version describes the record’s format. When the application moves from schema_version: 3 to a new format, the harness should either migrate old records deterministically or stop safely with an actionable recovery path. Quietly guessing at missing fields is a poor production strategy.

Observability should make one question easy to answer: why did this run take this action using this state? A useful trace can connect the session, checkpoint, model call, tool call, approval event, external identifier, and final outcome without exposing secrets.

Engineering principle

Resumability is a production property, not a UI feature. A “Continue” button is only trustworthy when the system can prove where execution stopped and what external effects already happened.

09Common Mistakes

1. Treating the whole transcript as the state.
The transcript is useful evidence, but it is not necessarily a reliable workflow record. Correction: store explicit workflow facts, approvals, external references, and checkpoint revisions.

2. Using a session ID as authorization.
A session identifier groups data; it does not establish that the caller is allowed to access that data. Correction: authenticate and authorize separately. The OpenAI Agents SDK documentation makes this distinction explicitly. See the storage trust-boundary notes.

3. Letting summaries become business truth.
A summary can omit a critical exception or preserve a malicious instruction. Correction: keep critical facts in typed, authoritative state and use summaries mainly as derived context.

4. Retrying an external write without reconciliation.
A timeout does not prove the action failed. Correction: record intent first, reconcile on recovery, then retry only under an idempotency or deduplication strategy the external system supports.

5. Persisting secrets inside memory.
Long-lived context tends to spread further than intended. Correction: keep credentials in dedicated secret storage and expose only the minimum authority needed at tool execution time.

6. Assuming compaction means “just summarize the chat.”
Long-running systems need to preserve the state required to continue correctly, not merely produce a shorter transcript. Provider compaction outputs can be opaque, so durable workflow facts should live in your own state rather than only inside a compacted window. See OpenAI’s compaction guidance.

7. Resuming two workers against the same mutable history.
Concurrent restoration can create duplicate or conflicting state transitions. Correction: use locking, optimistic concurrency on the revision, workflow ownership, or another explicit single-writer design appropriate to your architecture.

8. Checking only final output.
An agent can produce a convincing final sentence after making the wrong tool calls. Correction: evaluate state transitions, permissions, side effects, and recovery traces.

9. Trusting a snapshot because it deserializes cleanly.
A serialized run can carry approvals and pending tool calls. Correction: load snapshots only from trusted server-side storage, verify integrity and ownership, and never accept client-supplied state as authoritative.

🎯 Use this when...

You are reviewing an existing agent that “usually remembers,” but nobody can clearly explain how it recovers after failure.

10❓ FAQ

Q1. Is session history the same thing as agent memory?

No. Session history is one form of stored interaction state. Memory can include durable facts, summaries, preferences, artifacts, or other records. A harness should define which memories are authoritative and which are only contextual hints.

Q2. Does resuming a session mean the model will see everything from the previous run?

Not necessarily. A session can contain stored history while the harness selects, filters, summarizes, or compacts what is sent to the model. The OpenAI Agents SDK, for example, can cap how many stored items are retrieved for a run and lets you customize how history and new input are merged before the model call. See the session documentation.

Q3. What should happen when the last tool call may or may not have succeeded?

Treat the outcome as unknown until reconciled. Query an authoritative system when possible, use an idempotency or deduplication mechanism when supported, and stop for controlled review when the system cannot establish the outcome safely.

Q4. Should important state be written into the prompt so the model can remember it?

The model can use important state through its context, but the prompt should not become the authoritative database. Keep critical workflow facts in durable state and project the minimum necessary subset into the next model context.

Q5. What is the simplest state design for a small agent?

Start with a clearly scoped session record, a small workflow state object, explicit revision and schema versions, and a deterministic recovery path. Add compaction, distributed stores, or more advanced memory only when the workload actually requires them.

Q6. What is the difference between a checkpoint and a transcript?

A transcript records what was said and done in order. A checkpoint records where execution can safely restart: the stage reached, which external effects are confirmed, and what is pending. You can often rebuild a transcript view from a checkpoint plus history, but you should not have to read a transcript to work out whether an invoice was already sent.

REFERENCES🔗 References & Further Reading

Originality note.. Product names, APIs, and specifications belong to their respective owners.

SUMMARY📝 Summary

  • Session connects related runs; it is not automatically authorization.
  • Context is what the model sees for the current decision; it should be reconstructed deliberately.
  • Durable state records facts and workflow transitions that must survive restarts.
  • Checkpoints make recovery explicit instead of asking the model to guess what happened.
  • Safe resumption records intent before a consequential call, then reconciles uncertain external effects before retrying.
  • State is a security surface: scope it, protect its integrity, control retention, and track provenance.
  • Evaluate the harness, not just the answer: test state transitions, permissions, side effects, and recovery traces.

Comments