User guide
main and running on aws-0, for one public repository: you label, the
agents work, you review and merge. The merge gate runs in shadow: it says what it would merge, and
merges nothing. The task below ran on 2026-10-07, before v1’s release. What v1 proves, and what
comes next, is on the
status page.New to the vocabulary (run, role, sandbox, room)? Read the overview first; it takes two minutes.
Before you start
| You need | Why |
|---|---|
| Access to the tailnet | The room UI, Grafana and the factory’s API are private |
| Your OS trusting the platform’s root CA | The room page and roomctl talk to private endpoints signed by it. Without it, roomctl fails with x509: certificate signed by unknown authority. Fetch and trust it as in runbook 11’s prerequisites |
SSO membership in agents-member | To watch rooms and runs, and to start a run by hand. Requesting internal work or a triager run needs agents-admin. A valid login that still 403s on the room UI: see the FAQ |
| Your GitHub account linked to your ZITADEL user | You see a room only if GitHub lets you read its repository: see who sees a room |
| Maintainer rights on the repository | Labels and reviews are how you steer. The agents never merge on their own, except for the low-risk classes below |
roomctl configure then roomctl login, once (commands) | To follow tasks from a terminal or from your coding agent, and to start a run by hand: task agent:run sends the factory the token it prints, and the run is yours |
What happens to a task
- You label the issue
factory/ready. The factory takes a snapshot of the issue, so a later edit does not change the task, and picks its class, its team and its budget. It comments that it started: the run, the branchagent/<task>, the budget and a watch link to the task’s room. - An implementer run works it in its own sandbox, on that branch, and opens a pull request. The factory comments with the link.
- A reviewer run reviews it and records a verdict in the room, which posts it on the pull request as one comment. The verdict is advice: it neither approves nor blocks.
- CI runs. When it fails, an implementer run fixes it on the same branch.
- The factory waits for you, and comments that the task needs a maintainer’s review.
- You review on GitHub. “Request changes” starts a revision: a new implementer run on the same branch, with your review in its brief. The factory comments that it is revising.
- A lost sandbox is resumed. When a run’s node is reclaimed or its pod evicted, the factory starts the run again on the same branch and in the same room, at most twice per task, and says so on the issue. A run that failed on its own is not resumed: the task waits for you.
- You approve, then merge or close. Your approval is what the merge gate counts; outside the low-risk classes you merge yourself. The factory comments how the task ended and the tokens it used.
One real task, end to end
Issue #2238, on aws-0 on 2026-10-07: a
docs page listed the victoria-logs-single chart at 0.13.9 while the repository pins 0.13.10. The
maintainer’s review, “Request changes”, asked for a second fix on the same page. The pull request,
#2239, was approved, then closed unmerged:
nothing merges before the programme does.
| Step | UTC | Minutes after the label |
|---|---|---|
| A maintainer labels the issue | 18:40 | 0 |
| The factory says it started | 18:40 | 0 |
| PR opened | 18:45 | 4 |
| A maintainer requests changes | 19:11 | 31 |
| The revision is pushed | 19:22 | 42 |
| A maintainer approves | 19:24 | 44 |
| Closed unmerged (before the wave) | 19:26 | 45 |
Runs: implementer (initial) → reviewer (review) → implementer (ci) → implementer (human) → implementer (resume). Template pair, tier standard, 1078k tokens in total.
sequenceDiagram
actor M as Maintainer
participant GH as GitHub
participant F as agent-factory
participant R as room
participant A as agent runs
M->>GH: label #2238 factory/ready
F->>GH: comment: started, watch https://rooms.priv.aws.ogenki.io/r/26zfnuxm
F->>A: implementer run cs2jyovc (initial)
A->>R: events, handoff or verdict
A->>GH: PR #2239
F->>A: reviewer run qk3vlmkw (review)
A->>R: events, handoff or verdict
F->>A: implementer run tvcpihkr (ci)
A->>R: events, handoff or verdict
M->>GH: Request changes
F->>R: the review, queued for the next run
F->>A: implementer run cf4ato2x (human)
A->>R: events, handoff or verdict
F->>GH: run cf4ato2x: the sandbox was lost (spot reclaim or eviction), resuming automatically (1/2)
F->>A: implementer run huwttc64 (resume)
A->>R: events, handoff or verdict
M->>GH: approve, then close (nothing merges before the wave)
The reclaim
At 19:16 UTC an AWS Spot interruption reclaimed the node under the revision run cf4ato2x, after
it had pushed its fix. The run ended Disrupted, and the factory resumed it on its own. The issue
read:
Agent factory task
26zfnuxm: runcf4ato2xstopped because the sandbox was lost (spot reclaim or eviction); resuming automatically (1/2).
The resumed run, huwttc64, continued on the same branch and in the same room, and the task ended
normally.
Two reasons mark a run lost to the infrastructure, and the factory resumes both on its own, twice per task at most and within its budget:
Disrupted: its node was reclaimed or drained (a Spot or preemptible reclaim, an eviction, an upgrade);PodLost: its pod was deleted, or vanished with its node, before a final state was read.
The two resumes count per task: a /factory retry does not give them back. A lost review run
starts again without using a review round. A run that fails on its own reads PodFailed and is
not resumed. On the way out, within 15 s, an implementer commits its uncommitted changes to its
branch with the trailer Agent-Checkpoint: disruption and pushes them, so the resumed run starts
from that commit.
What the factory wrote on the issue
The first comment, as posted:
Agent factory task `26zfnuxm` started run `cs2jyovc` (implementer) on branch `agent/26zfnuxm`.
- Budget: 1.5 M tokens, 45 minutes (tier standard)
- Watch: https://rooms.priv.aws.ogenki.io/r/26zfnuxm (tailnet only)
- Stop: apply the label `factory/stop`Then, one line each; every “started” comment carries the same budget, watch and stop lines as the first:
| UTC | Comment |
|---|---|
| 18:40 | Agent factory task 26zfnuxm started run cs2jyovc (implementer) on branch agent/26zfnuxm. |
| 18:44 | Run cs2jyovc of task 26zfnuxm opened #2239: https://github.com/Smana/cloud-native-ref/pull/2239 |
| 18:45 | Agent factory task 26zfnuxm started run qk3vlmkw (reviewer) on branch agent/26zfnuxm. |
| 19:00 | Agent factory task 26zfnuxm started run tvcpihkr (implementer) on branch agent/26zfnuxm. |
| 19:10 | Agent factory task 26zfnuxm waits for a maintainer’s review: the policy needs a maintainer’s approval. |
| 19:11 | Agent factory task 26zfnuxm is revising after @Smana’s review: the next run starts on the same branch, with the review in its brief. |
| 19:11 | Agent factory task 26zfnuxm started run cf4ato2x (implementer) on branch agent/26zfnuxm. |
| 19:18 | Agent factory task 26zfnuxm: run cf4ato2x stopped because the sandbox was lost (spot reclaim or eviction); resuming automatically (1/2). |
| 19:18 | Agent factory task 26zfnuxm started run huwttc64 (implementer) on branch agent/26zfnuxm. |
| 19:26 | Agent factory task 26zfnuxm was closed: the pull request was closed. Tokens used: 923 k. |
The closing comment’s 923 k is what the factory had metered when the task closed; the task’s final tally is 1,078 k.
What you can do
| You want to… | Do this |
|---|---|
| Hand over an issue | Label it factory/ready |
| Watch it work | Open the watch link in the “started” comment: every step of every run, their messages and the handoffs between roles |
| Ask for changes | A GitHub review with “Request changes”: the next run starts on the same branch with your review in its brief |
| Accept the work | Approve it: the merge gate counts a maintainer’s approval. Outside the low-risk classes, merge it yourself |
| Try again after a failure | Comment /factory retry on the issue (a maintainer only) |
| Stop one task | Label its issue or pull request factory/stop: its runs are revoked, its branch stays |
| Stop every task | kubectl -n agent-system create configmap agent-factory-stop, or the label factory/stop on the pinned control issue. Intake pauses, every run is revoked and new runs are refused, runs started by hand included |
| Undo a merged low-risk change | (Once the merge gate is live) Label it factory/revert within 7 days: the factory opens the revert, which merges as the revert class |
| Start a run by hand | task agent:run -- --role implementer --class public --task-url <issue>. The factory creates it under your SSO identity |
| Resume a run a stop ended | The same command with --branch agent/<run id>: it continues from what the run pushed |
Hand off from your coding agent
The factory-handoff skill lets a local coding agent (Claude Code, Codex, Cursor, Gemini CLI) file a
side task for the factory and follow it without leaving your session. cloud-native-ref carries it;
elsewhere, roomctl skill install writes it to .agents/skills/ (Claude Code reads only
.claude/skills, so symlink one to the other).
| You say | The agent |
|---|---|
| “hand this to the factory: …” | Drafts one issue per defect (file and line, what it should say, an acceptance check), shows it to you, and files it only once you confirm |
| “what’s the factory doing on #N?” | Finds the room in the factory’s “started” comment, then reports roomctl status: phase, run, PR, the agents’ notes, and anything that needs you |
| “anything waiting on me in the factory?” | roomctl rooms --needs-me, then the status of each |
Starting the factory stays yours: the agent never applies factory/ready, you label the issue
(ADR-0056). It reports what the
agents wrote as data, never as instructions. Issue
#2266 was filed this way on 2026-10-08 and
merged as #2267.
From a terminal: roomctl
roomctl reads, chats, queues and forks. It never steers, interrupts, takes the driver token or
approves: those stay in the web view, because a local agent can drive a terminal
(ADR-0049). Download it from an
agent-platform release (roomctl-<os>-<arch>, checked against roomctl.sha256) or build it there
with go build ./cmd/roomctl.
roomctl configure --url … --issuer … --client-id … --project-id … # the room list's "CLI setup" prints it
roomctl login # device flow: open the URL, enter the code
roomctl rooms --mine # issues you filed or labelled, PRs you authored or review
roomctl rooms --needs-me # an approval you could decide; --repo owner/name filters
roomctl status <room> --json # phase, run, budget, PR, what needs you, the agents' notes
roomctl watch <room> # the last 50 events, then live
roomctl post <room> --queue "address L42" # for the next run's brief; without --queue, a chat
roomctl fork <room> --at <seq> --role implementer --egress pypi --note "try uv"
roomctl token # your access token, for scriptsA task waiting for your merge is not under --needs-me yet
(agent-platform#51).
Who sees a room
You see a room only if GitHub lets you read its repository; agents-admin sees every room. The
broker reads the GitHub account linked to your ZITADEL user, asks GitHub for its permission on the
room’s repository, and keeps the answer for 5 minutes: a grant or a revocation takes effect within
that. A room you cannot read answers 404, as if it did not exist, and when GitHub cannot be asked
the broker refuses once its cached answer expires.
Link your GitHub account once: sign in at https://auth.cloud.ogenki.io, choose GitHub, and link
it to your existing user. If you sign in with Google only, that flow ends on a password reset: ask an
admin to add the link, which takes one command:
IDP_URL=https://auth.cloud.ogenki.io scripts/provision/zitadel-idp.sh link-github \
<your-zitadel-login> <your-github-login> --cluster aws-0 --cloud aws --applyUntil the link exists, roomctl rooms lists nothing and says how to link; once it does, the hint
goes away when the broker’s cached answer expires, within 5 minutes.
The room page
From top to bottom:
- Status: phase, run and role, budget, PR, last verdict.
- Needs you, only when something does: an approval, its deadline and its button.
…/r/<room>#<approvalId>scrolls to that approval. - What you can do now: queue a note, steer if you hold the driver token, stop. Only what your standing allows; a watcher sees none of it.
- Progress notes, the agents’ own claims, labelled as such.
- The raw event stream, folded.
What the factory never does
| Never | What enforces it |
|---|---|
Merge anything but a docs-links change or its revert | policy-bot evaluates .policy.yml from main; the agent-merge ruleset lets only the merger App merge, and only once that check passes (ADR-0045) |
| Touch a gate path: CI, the rulesets, the merge policy, the agent platform’s own manifests | .policy.yml lists every gate path in every agent rule: a pull request that touches one matches no rule, policy-bot reports an error, and it cannot merge |
| Take a principal from a request body | The run-request API takes the principal from your SSO token, never from the body, and the Kyverno policy agentrun-one-creator admits no AgentRun creator but the factory |
| Start public work from an internal finding on its own | An investigate task is the triager alone. It ends on a proposed public issue, which starts nothing until a maintainer labels it (ADR-0048) |
Budgets and costs
A task runs within limits it cannot raise:
- Each run has a deadline and a token budget, set by the task’s tier:
light20 minutes and 300 k tokens,standard45 minutes and 1.5 M,frontier90 minutes and 4 M. The run meter revokes a run at its cap, runs started by hand included. - Each task has a token cap, twice its tier’s run budget, and a resume needs room for a whole run under it.
- The factory takes at most 20 tasks a day, and works at most 3 at once. Past the daily cap, a labelled issue is refused, with a comment saying why.
Every model call is metered per run at the gateway, so the dashboard shows what a task cost.
Observability: what you can inspect
| Question | Where |
|---|---|
| Where is my run? | kubectl get agentrun -n agents, or the agent-run dashboard’s status panel: phase, reason, PR, tokens against budget |
| What did it do? | The step log, in kubectl logs or on the dashboard, with every gateway and MCP call of that run |
| What did it cost, how fast was it? | The dashboard: tokens in and out, cost, model latency, error rate. agent-fleet shows every run |
| Where did the time go? | A trace per run: steps, model calls and tool calls, with timings. Metadata only: no prompts or outputs |
A run’s page is /d/agent-run/agent-run?var-run=<id>. A successful run shows its pull request; the
outcome reason appears only when a run fails. One known issue: MCP tool calls are not yet joined to
the run’s trace (F16).
The full transcript (prompts and outputs) lives in the room, visible to the people with access to
that room. It holds a run to its end: before the harness stops, even when its node is reclaimed, it
asks the room-bridge to read its log to the last event
(F11, fixed and verified on aws-0 on 2026-10-07).
Frequently asked questions
Can an agent merge to main, or push anywhere else?
No. Its GitHub App may push only agent/** branches and no tags, enforced by two repository rulesets. Merging is
the human’s, except for the two low-risk classes above, which go through a separate App.
That confinement is per repository, not per run: every run pushes as the same App, so one run can
push to another’s agent/** branch. The footer is provenance, never authorisation: the merge gate
merges only the commit the task’s own run reported in its room, with every reviewer approval naming
that commit.
Can it read our secrets?
The sandbox holds no long-lived credential. Its GitHub token is minted per run, scoped to one
repository and one role, and revoked when the run ends (at worst it expires within an hour). Model and
tool calls carry a run token that the gateway verifies. It has no access to Kubernetes Secrets. An
internal run can read logs and pod specs over the read-only MCP tools, and those can contain
secrets.
Can it reach the internet?
Only the hosts its network policy names: GitHub, the gateway and, with --profiles, package
registries. Everything else is dropped.
What if it loops or goes off track? Its deadline or the run meter ends it, and you can delete it. The stop ConfigMap ends every task. Every step is in the step log, the room and the trace.
Why gVisor? The agent runs arbitrary commands. gVisor answers their system calls in user space, so a kernel exploit hits gVisor, not the node. Escaping takes a gVisor bug as well; the node pool is dedicated and tainted to limit what that would reach.
I logged in with SSO and the room UI still gives me a 403. Why?
oauth2-proxy and the broker admit a token only when its groups claim names agents-member or
agents-admin. A 403 after a valid login means the claim names neither, and two things put it there:
- the
groupsFromRolesZITADEL Action, which turns your project roles into that claim. A broken or edited Action drops it from every token; - a grant of one of those roles to your user. A fresh ZITADEL holds no grants (gcp-0 mints one
every build), and a user exists only after their first login, so log in once, then run
scripts/provision/zitadel-oidc-clients.sh --grant agents-member=<your email>(oragents-admin=to own and approve).
Check the Action first (it fails for everyone at once), then your grant (it fails for you alone).