Harness tuning

The harness runs the agent loop inside the sandbox: OpenHands agent-server, driven by a small entrypoint, agent-run. This page is the guide; every setting with its value and bounds is on the Harness reference.

Start here

The five levers

How a run works

agent-run starts agent-server, clones the repository, hands it the task, then follows it until the conversation ends.

    sequenceDiagram
  autonumber
  participant AR as agent-run
  participant AS as agent-server
  participant LLM as agent-router
  participant GH as GitHub
  AR->>AS: start, wait until ready
  AR->>GH: clone the branch
  AR->>AS: start the conversation (task, rules, tools)
  loop at most 500 iterations
    AS->>LLM: model call (300 s timeout)
    LLM-->>AS: tool calls
    AS->>AS: run the tools, condense history past 80 events
  end
  AS-->>AR: finished | error | stuck
  AR->>GH: revoke the run's token
  Note over AR: exit 0 on finished, 1 otherwise
  

The tester’s run

A tester run writes tests under the repository’s test paths and commits them. The harness then runs them red on the base, signs the commit and pushes it, or answers once (not_testable) and pushes nothing. The implementer that follows cannot commit a change to those files, and is told when they still fail. The model itself never pushes. testFirst is off by default; an operator turns it on per class (runbook 21), and a class without it runs without a tester (the roles).

agent-run step 3: terminal | List crossplane controller dir and kustomization | ls /workspace/repo/... | trace_id=706f…
agent-run model retry 1/2 in 10s: ServiceUnavailableError …
agent-run: conversation ended with execution_status=finished
agent-run summary: 21 steps, 0 model retries

Where settings come from

Each layer lives in its own repository and is pinned into the next. A run’s settings are fixed when it is created; only its token cap can be raised while it runs.

    flowchart TB
  subgraph decide["Config and claim"]
    direction LR
    F["Factory config<br/>tiers · templates · caps"]:::cfg --> T["Triage<br/>tier + template"]:::cfg
    API["Run API<br/>task agent:run"]:::cfg --> C
    T --> C["AgentRun claim<br/>role · model · budget"]:::claim
  end
  subgraph run["The run"]
    direction LR
    K["Composition<br/>pod · env · egress"]:::claim --> H["Harness image<br/>loop settings"]:::run
    H -- every call --> G["Gateway<br/>models · MCP · budgets"]:::gw
  end
  decide -->|"the claim composes"| run
  classDef cfg fill:#6366f1,stroke:#4f46e5,color:#fff
  classDef claim fill:#059669,stroke:#047857,color:#fff
  classDef run fill:#f59e0b,stroke:#d97706,color:#111
  classDef gw fill:#7c3aed,stroke:#6d28d9,color:#fff
  

Trade-offs

Measuring a change

Change one lever, run the same kind of task, compare. A run’s id is the 8 characters after xplane-run-.

# The step log, then the tool mix
kubernetes.pod_name:"xplane-run-<id>" AND kubernetes.container_name:"harness" AND _msg:~"^agent-run"
kubernetes.pod_name:"xplane-run-<id>" AND kubernetes.container_name:"harness" AND _msg:~"^agent-run step "
  | extract "agent-run step <n>: <tool> |" | stats by (tool) count() steps

# Cost, as the meter computes it
maku_factory_run_spend_usd{run="<id>"}
# Was the tier right? fit = under | fit | over
sum by (tier, fit) (maku_factory_tier_fit_total)
# Why runs end
sum by (reason) (agentrun_outcome_info)

What real runs look like

Example (the reference host): typical runs, all on GLM-5.3.

Failure modes

Budget and time

Loops and errors

The tester’s contract

Environment and rooms

Recipes

Debug a run that loops

    flowchart TB
  A["Run long<br/>or failed"]:::start --> B{"execution<br/>status?"}
  B -- stuck --> S["Fix the repeated<br/>tool or host"]:::fix
  B -- error --> E{"Error line?"}
  E -- "max iterations" --> L
  E -- unavailable --> P["Retry the task"]:::fix
  E -- timeout --> R["Check<br/>reasoning_effort"]:::fix
  B -- "revoked or<br/>deadline" --> L{"Step mix?"}
  L -- "reads, no commit" --> W["Scope the task"]:::fix
  L -- "commits, pushes" --> G["More runUSD<br/>+ runMinutes"]:::fix
  L -- "refused in a room" --> Q["Approve, or fork<br/>with egress"]:::fix
  classDef start fill:#dc2626,stroke:#b91c1c,color:#fff
  classDef fix fill:#059669,stroke:#047857,color:#fff
  

1. kubectl get agentrun -n agents xplane-run-<id>: phase and reason.
2. The step log’s last 30 lines: repeating, or changing?
3. The tool mix (the LogsQL tab above): 95% terminal and no commit is searching, not working.
4. The room transcript for the model’s reasoning; the trace for timings.
5. If it burns money, stop it: factory/stop on the issue.

Loop policy

The loop policy stops wandering runs and checks a finished run against the task’s acceptance checks. An impossible task escalates as “cannot be done as asked”, never as a false met, and a failing named check is never met. A class without a row behaves as before.

The switch is the factory’s loops key, one row per class (default, review, or a class a repository declares). A row that sets any of its three keys switches that class on. No row, or a row without them, runs with no loop policy. Example (the reference host): docs and scripts set them; scripts has judge: tier-light, so the defaults apply (a stall at 80 steps, 2 continuations), and default: {scope: off} switches nothing on.

The factory copies the row into the claim as spec.loop, on every implementer claim of the task: the first run, revisions, CI fixes, retries and resumes. Reviewers, testers and triagers never carry it. The composition (crossplane-configuration v0.9.18) hands it to agent-run as AGENT_LOOP, a JSON object with stallSteps, continuations and judgeModel. A cluster whose composition is older refuses the claim, and the task escalates with loop_unsupported.

The three ends. met is not an end: the run finishes and the task goes to the next role.

The counter also reads finished for a run under the policy that ended normally, and other for any end not listed above, by class.

A pending check is waited for. A not_yet whose only unmet check is still running waits for the check inside the 15-minute box, and spends no continuation. When a check fails, the judge is shown the log’s last 60 lines, for up to 3 jobs. That quote is empty unless the harness’s egress allows the Actions log host (productionresultssa*.blob.core.windows.net); allowing it is a security decision for your platform.

Switching a class on is last: add its row only after you have tried it on one class. The meter and the factory’s outcomes then say whether cost per accepted pull request fell.

Search

Experimental. The implementer can ask a read-only helper one question about the repository instead of reading it file by file. Its benefit is unmeasured. Example (the reference host): only docs has search: true, so every other run behaves as before.

    %%{init: {"sequence": {"actorMargin": 20, "width": 120, "messageMargin": 25}}}%%
sequenceDiagram
  autonumber
  participant I as implementer<br/>(agent-server)
  participant S as search tool<br/>(same process)
  participant C as child<br/>conversation
  participant P as identity<br/>proxy /v1
  participant G as gateway<br/>(meter, caps)
  participant H as agent-run
  I->>S: search(question)
  S->>S: call cap from the ledger<br/>one at a time
  S->>C: fresh context:<br/>search brief + question<br/>grep, glob, read_file,<br/>list_directory
  loop at most 40 iterations, 300k tokens, 300 s
    C->>P: chat completion<br/>(the run's model,<br/>effort low)
    P->>G: the run's token<br/>x-ar-model pinned
    G-->>C: metered on the run,<br/>inside its caps
    C->>C: read-only tool calls<br/>in /workspace/repo
  end
  C-->>S: final message,<br/>or a stop with<br/>its partial answer
  S->>S: ledger row · header<br/>clip at 8,000 characters
  S-->>I: observation<br/>(an error observation<br/>on a failure)
  H->>H: reads the ledger:<br/>the child's steps<br/>join the stall count
  

The child sees nothing of the conversation, reads the uncommitted tree, never writes and never reaches the room. Its calls are metered on the run and bounded by the run’s USD cap, and the bridge classes the call plain. The switch is loops.<class>.search: true (reference), which does not switch the loop policy on.

Not tunable today

Related: Harness reference · Factory · Rooms · Observability · User guide