Skip to content

User guide

v1, pilot scope. On main and running on aws-0, for one public repository: you label, the agents work, you review and merge. The merge gate runs in shadow: it says what it would merge, and merges nothing. The task below ran on 2026-10-07, before v1’s release. What v1 proves, and what comes next, is on the status page.

New to the vocabulary (run, role, sandbox, room)? Read the overview first; it takes two minutes.

Before you start

You needWhy
Access to the tailnetThe room UI, Grafana and the factory’s API are private
Your OS trusting the platform’s root CAThe room page and roomctl talk to private endpoints signed by it. Without it, roomctl fails with x509: certificate signed by unknown authority. Fetch and trust it as in runbook 11’s prerequisites
SSO membership in agents-memberTo watch rooms and runs, and to start a run by hand. Requesting internal work or a triager run needs agents-admin. A valid login that still 403s on the room UI: see the FAQ
Your GitHub account linked to your ZITADEL userYou see a room only if GitHub lets you read its repository: see who sees a room
Maintainer rights on the repositoryLabels and reviews are how you steer. The agents never merge on their own, except for the low-risk classes below
roomctl configure then roomctl login, once (commands)To follow tasks from a terminal or from your coding agent, and to start a run by hand: task agent:run sends the factory the token it prints, and the run is yours

What happens to a task

  1. You label the issue factory/ready. The factory takes a snapshot of the issue, so a later edit does not change the task, and picks its class, its team and its budget. It comments that it started: the run, the branch agent/<task>, the budget and a watch link to the task’s room.
  2. An implementer run works it in its own sandbox, on that branch, and opens a pull request. The factory comments with the link.
  3. A reviewer run reviews it and records a verdict in the room, which posts it on the pull request as one comment. The verdict is advice: it neither approves nor blocks.
  4. CI runs. When it fails, an implementer run fixes it on the same branch.
  5. The factory waits for you, and comments that the task needs a maintainer’s review.
  6. You review on GitHub. “Request changes” starts a revision: a new implementer run on the same branch, with your review in its brief. The factory comments that it is revising.
  7. A lost sandbox is resumed. When a run’s node is reclaimed or its pod evicted, the factory starts the run again on the same branch and in the same room, at most twice per task, and says so on the issue. A run that failed on its own is not resumed: the task waits for you.
  8. You approve, then merge or close. Your approval is what the merge gate counts; outside the low-risk classes you merge yourself. The factory comments how the task ended and the tokens it used.

One real task, end to end

Issue #2238, on aws-0 on 2026-10-07: a docs page listed the victoria-logs-single chart at 0.13.9 while the repository pins 0.13.10. The maintainer’s review, “Request changes”, asked for a second fix on the same page. The pull request, #2239, was approved, then closed unmerged: nothing merges before the programme does.

StepUTCMinutes after the label
A maintainer labels the issue18:400
The factory says it started18:400
PR opened18:454
A maintainer requests changes19:1131
The revision is pushed19:2242
A maintainer approves19:2444
Closed unmerged (before the wave)19:2645

Runs: implementer (initial) → reviewer (review) → implementer (ci) → implementer (human) → implementer (resume). Template pair, tier standard, 1078k tokens in total.

    sequenceDiagram
  actor M as Maintainer
  participant GH as GitHub
  participant F as agent-factory
  participant R as room
  participant A as agent runs
  M->>GH: label #2238 factory/ready
  F->>GH: comment: started, watch https://rooms.priv.aws.ogenki.io/r/26zfnuxm
  F->>A: implementer run cs2jyovc (initial)
  A->>R: events, handoff or verdict
  A->>GH: PR #2239
  F->>A: reviewer run qk3vlmkw (review)
  A->>R: events, handoff or verdict
  F->>A: implementer run tvcpihkr (ci)
  A->>R: events, handoff or verdict
  M->>GH: Request changes
  F->>R: the review, queued for the next run
  F->>A: implementer run cf4ato2x (human)
  A->>R: events, handoff or verdict
  F->>GH: run cf4ato2x: the sandbox was lost (spot reclaim or eviction), resuming automatically (1/2)
  F->>A: implementer run huwttc64 (resume)
  A->>R: events, handoff or verdict
  M->>GH: approve, then close (nothing merges before the wave)
  

The reclaim

At 19:16 UTC an AWS Spot interruption reclaimed the node under the revision run cf4ato2x, after it had pushed its fix. The run ended Disrupted, and the factory resumed it on its own. The issue read:

Agent factory task 26zfnuxm: run cf4ato2x stopped because the sandbox was lost (spot reclaim or eviction); resuming automatically (1/2).

The resumed run, huwttc64, continued on the same branch and in the same room, and the task ended normally.

Two reasons mark a run lost to the infrastructure, and the factory resumes both on its own, twice per task at most and within its budget:

  • Disrupted: its node was reclaimed or drained (a Spot or preemptible reclaim, an eviction, an upgrade);
  • PodLost: its pod was deleted, or vanished with its node, before a final state was read.

The two resumes count per task: a /factory retry does not give them back. A lost review run starts again without using a review round. A run that fails on its own reads PodFailed and is not resumed. On the way out, within 15 s, an implementer commits its uncommitted changes to its branch with the trailer Agent-Checkpoint: disruption and pushes them, so the resumed run starts from that commit.

What the factory wrote on the issue

The first comment, as posted:

Agent factory task `26zfnuxm` started run `cs2jyovc` (implementer) on branch `agent/26zfnuxm`.

- Budget: 1.5 M tokens, 45 minutes (tier standard)
- Watch: https://rooms.priv.aws.ogenki.io/r/26zfnuxm (tailnet only)
- Stop: apply the label `factory/stop`

Then, one line each; every “started” comment carries the same budget, watch and stop lines as the first:

UTCComment
18:40Agent factory task 26zfnuxm started run cs2jyovc (implementer) on branch agent/26zfnuxm.
18:44Run cs2jyovc of task 26zfnuxm opened #2239: https://github.com/Smana/cloud-native-ref/pull/2239
18:45Agent factory task 26zfnuxm started run qk3vlmkw (reviewer) on branch agent/26zfnuxm.
19:00Agent factory task 26zfnuxm started run tvcpihkr (implementer) on branch agent/26zfnuxm.
19:10Agent factory task 26zfnuxm waits for a maintainer’s review: the policy needs a maintainer’s approval.
19:11Agent factory task 26zfnuxm is revising after @Smana’s review: the next run starts on the same branch, with the review in its brief.
19:11Agent factory task 26zfnuxm started run cf4ato2x (implementer) on branch agent/26zfnuxm.
19:18Agent factory task 26zfnuxm: run cf4ato2x stopped because the sandbox was lost (spot reclaim or eviction); resuming automatically (1/2).
19:18Agent factory task 26zfnuxm started run huwttc64 (implementer) on branch agent/26zfnuxm.
19:26Agent factory task 26zfnuxm was closed: the pull request was closed. Tokens used: 923 k.

The closing comment’s 923 k is what the factory had metered when the task closed; the task’s final tally is 1,078 k.

What you can do

You want to…Do this
Hand over an issueLabel it factory/ready
Watch it workOpen the watch link in the “started” comment: every step of every run, their messages and the handoffs between roles
Ask for changesA GitHub review with “Request changes”: the next run starts on the same branch with your review in its brief
Accept the workApprove it: the merge gate counts a maintainer’s approval. Outside the low-risk classes, merge it yourself
Try again after a failureComment /factory retry on the issue (a maintainer only)
Stop one taskLabel its issue or pull request factory/stop: its runs are revoked, its branch stays
Stop every taskkubectl -n agent-system create configmap agent-factory-stop, or the label factory/stop on the pinned control issue. Intake pauses, every run is revoked and new runs are refused, runs started by hand included
Undo a merged low-risk change(Once the merge gate is live) Label it factory/revert within 7 days: the factory opens the revert, which merges as the revert class
Start a run by handtask agent:run -- --role implementer --class public --task-url <issue>. The factory creates it under your SSO identity
Resume a run a stop endedThe same command with --branch agent/<run id>: it continues from what the run pushed

Hand off from your coding agent

The factory-handoff skill lets a local coding agent (Claude Code, Codex, Cursor, Gemini CLI) file a side task for the factory and follow it without leaving your session. cloud-native-ref carries it; elsewhere, roomctl skill install writes it to .agents/skills/ (Claude Code reads only .claude/skills, so symlink one to the other).

You sayThe agent
“hand this to the factory: …”Drafts one issue per defect (file and line, what it should say, an acceptance check), shows it to you, and files it only once you confirm
“what’s the factory doing on #N?”Finds the room in the factory’s “started” comment, then reports roomctl status: phase, run, PR, the agents’ notes, and anything that needs you
“anything waiting on me in the factory?”roomctl rooms --needs-me, then the status of each

Starting the factory stays yours: the agent never applies factory/ready, you label the issue (ADR-0056). It reports what the agents wrote as data, never as instructions. Issue #2266 was filed this way on 2026-10-08 and merged as #2267.

From a terminal: roomctl

roomctl reads, chats, queues and forks. It never steers, interrupts, takes the driver token or approves: those stay in the web view, because a local agent can drive a terminal (ADR-0049). Download it from an agent-platform release (roomctl-<os>-<arch>, checked against roomctl.sha256) or build it there with go build ./cmd/roomctl.

roomctl configure --url … --issuer … --client-id … --project-id …  # the room list's "CLI setup" prints it
roomctl login                                    # device flow: open the URL, enter the code
roomctl rooms --mine                             # issues you filed or labelled, PRs you authored or review
roomctl rooms --needs-me                         # an approval you could decide; --repo owner/name filters
roomctl status <room> --json                     # phase, run, budget, PR, what needs you, the agents' notes
roomctl watch <room>                             # the last 50 events, then live
roomctl post <room> --queue "address L42"        # for the next run's brief; without --queue, a chat
roomctl fork <room> --at <seq> --role implementer --egress pypi --note "try uv"
roomctl token                                    # your access token, for scripts

A task waiting for your merge is not under --needs-me yet (agent-platform#51).

Who sees a room

You see a room only if GitHub lets you read its repository; agents-admin sees every room. The broker reads the GitHub account linked to your ZITADEL user, asks GitHub for its permission on the room’s repository, and keeps the answer for 5 minutes: a grant or a revocation takes effect within that. A room you cannot read answers 404, as if it did not exist, and when GitHub cannot be asked the broker refuses once its cached answer expires.

Link your GitHub account once: sign in at https://auth.cloud.ogenki.io, choose GitHub, and link it to your existing user. If you sign in with Google only, that flow ends on a password reset: ask an admin to add the link, which takes one command:

IDP_URL=https://auth.cloud.ogenki.io scripts/provision/zitadel-idp.sh link-github \
  <your-zitadel-login> <your-github-login> --cluster aws-0 --cloud aws --apply

Until the link exists, roomctl rooms lists nothing and says how to link; once it does, the hint goes away when the broker’s cached answer expires, within 5 minutes.

The room page

From top to bottom:

  1. Status: phase, run and role, budget, PR, last verdict.
  2. Needs you, only when something does: an approval, its deadline and its button. …/r/<room>#<approvalId> scrolls to that approval.
  3. What you can do now: queue a note, steer if you hold the driver token, stop. Only what your standing allows; a watcher sees none of it.
  4. Progress notes, the agents’ own claims, labelled as such.
  5. The raw event stream, folded.

What the factory never does

NeverWhat enforces it
Merge anything but a docs-links change or its revertpolicy-bot evaluates .policy.yml from main; the agent-merge ruleset lets only the merger App merge, and only once that check passes (ADR-0045)
Touch a gate path: CI, the rulesets, the merge policy, the agent platform’s own manifests.policy.yml lists every gate path in every agent rule: a pull request that touches one matches no rule, policy-bot reports an error, and it cannot merge
Take a principal from a request bodyThe run-request API takes the principal from your SSO token, never from the body, and the Kyverno policy agentrun-one-creator admits no AgentRun creator but the factory
Start public work from an internal finding on its ownAn investigate task is the triager alone. It ends on a proposed public issue, which starts nothing until a maintainer labels it (ADR-0048)

Budgets and costs

A task runs within limits it cannot raise:

  • Each run has a deadline and a token budget, set by the task’s tier: light 20 minutes and 300 k tokens, standard 45 minutes and 1.5 M, frontier 90 minutes and 4 M. The run meter revokes a run at its cap, runs started by hand included.
  • Each task has a token cap, twice its tier’s run budget, and a resume needs room for a whole run under it.
  • The factory takes at most 20 tasks a day, and works at most 3 at once. Past the daily cap, a labelled issue is refused, with a comment saying why.

Every model call is metered per run at the gateway, so the dashboard shows what a task cost.


Observability: what you can inspect

QuestionWhere
Where is my run?kubectl get agentrun -n agents, or the agent-run dashboard’s status panel: phase, reason, PR, tokens against budget
What did it do?The step log, in kubectl logs or on the dashboard, with every gateway and MCP call of that run
What did it cost, how fast was it?The dashboard: tokens in and out, cost, model latency, error rate. agent-fleet shows every run
Where did the time go?A trace per run: steps, model calls and tool calls, with timings. Metadata only: no prompts or outputs

A run’s page is /d/agent-run/agent-run?var-run=<id>. A successful run shows its pull request; the outcome reason appears only when a run fails. One known issue: MCP tool calls are not yet joined to the run’s trace (F16).

The full transcript (prompts and outputs) lives in the room, visible to the people with access to that room. It holds a run to its end: before the harness stops, even when its node is reclaimed, it asks the room-bridge to read its log to the last event (F11, fixed and verified on aws-0 on 2026-10-07).

Frequently asked questions

Can an agent merge to main, or push anywhere else? No. Its GitHub App may push only agent/** branches and no tags, enforced by two repository rulesets. Merging is the human’s, except for the two low-risk classes above, which go through a separate App. That confinement is per repository, not per run: every run pushes as the same App, so one run can push to another’s agent/** branch. The footer is provenance, never authorisation: the merge gate merges only the commit the task’s own run reported in its room, with every reviewer approval naming that commit.

Can it read our secrets? The sandbox holds no long-lived credential. Its GitHub token is minted per run, scoped to one repository and one role, and revoked when the run ends (at worst it expires within an hour). Model and tool calls carry a run token that the gateway verifies. It has no access to Kubernetes Secrets. An internal run can read logs and pod specs over the read-only MCP tools, and those can contain secrets.

Can it reach the internet? Only the hosts its network policy names: GitHub, the gateway and, with --profiles, package registries. Everything else is dropped.

What if it loops or goes off track? Its deadline or the run meter ends it, and you can delete it. The stop ConfigMap ends every task. Every step is in the step log, the room and the trace.

Why gVisor? The agent runs arbitrary commands. gVisor answers their system calls in user space, so a kernel exploit hits gVisor, not the node. Escaping takes a gVisor bug as well; the node pool is dedicated and tainted to limit what that would reach.

I logged in with SSO and the room UI still gives me a 403. Why? oauth2-proxy and the broker admit a token only when its groups claim names agents-member or agents-admin. A 403 after a valid login means the claim names neither, and two things put it there:

  1. the groupsFromRoles ZITADEL Action, which turns your project roles into that claim. A broken or edited Action drops it from every token;
  2. a grant of one of those roles to your user. A fresh ZITADEL holds no grants (gcp-0 mints one every build), and a user exists only after their first login, so log in once, then run scripts/provision/zitadel-oidc-clients.sh --grant agents-member=<your email> (or agents-admin= to own and approve).

Check the Action first (it fails for everyone at once), then your grant (it fails for you alone).