My dark factory: GitHub board cards that turn into pull requests

A dark factory is a factory that runs with the lights off: no people on the floor. Mine builds software. I file an issue on a GitHub project board, approve two steps, and an AI agent running in my Kubernetes cluster opens the pull request.

It runs on several boards: the HRB hosting platform, Samma, GoodTribes, kickfix, asylguiden and the Elino mail suite. This post shows how it's built and, more interesting, what broke when it met the real world.

The flow

The board's Status column is the whole state machine:

new ──► request ──► [human] ──► plan ──► review ──► [human] ──► apply ──► test ──► done
        (outline)                        (plan)                 (agent → PR)

Stage	Who	What happens
new → request	worker	Reads the issue and the live repository, posts a Request Outline as a comment
request → plan	me	I read the outline and move the card if I agree
plan → review	worker	Expands the outline into an Implementation Plan, max 12 steps
review → apply	me	I approve the plan
apply → test	worker	Hands the plan to an agent, which clones the repo, makes the change and opens a PR

The two human steps are hard rules: the board client refuses to make those two moves, and a test runs every stage against every column to prove it.

The parts

organizzer is one small Go binary with three modes: request, plan and apply. Each mode runs as its own Deployment with replicas: 1 and loops:

list the board
  → keep cards in my column (skip closed, needs-human, picked-by-*, recently failed)
  → take ONE card
  → post the artifact as a comment, then move the card
  → sleep 10 s (60 s if there was nothing to do)

kube-foundry is an operator that runs the agent. The apply worker creates a SoftwareTask, and the operator starts a sandbox pod with the OpenCode agent. The agent clones the repo, works on a factory/<task> branch, runs the build and opens the PR.

apiVersion: factory.factory.io/v1alpha1
kind: SoftwareTask
metadata:
  name: site-add-contact-page        # becomes the branch name
  namespace: kubefoundry
spec:
  repo: https://github.com/<org>/<repo>
  branch: main
  agent: open-code
  skills: [berget]                   # provider + default model
  credentials:
    secretRef: factory-creds-<project>
  resources:
    cpu: "2"
    memory: 4Gi
    timeoutMinutes: 30
  task: |
    Add a contact page ...

    - Read the relevant files before changing anything.
    - Keep the change small: only the pages and nav.
    - In the PR description, list the files you changed and why.

The models come from berget.ai, an EU-hosted, OpenAI-compatible API. A small, fast model writes the outlines, a reasoning model writes the plans, and the agent's model is picked per task with a "skill" (berget-fast for simple edits, berget-large for hard ones).

Rules that keep it safe

• The status field is the lock. There is no database and no locking, so each stage runs exactly one replica. And never run the old manual scripts at the same time as the workers.
• Comment first, then move. The card moves only after the outline or plan is posted. If the worker crashes in between, the card is still in its input column and the work is redone cheaply. The other order can leave a card waiting for approval of something that was never posted.
• Sentinels. Every artifact ends with a fixed marker line. Each stage checks for its own marker to know if it already ran. For the apply stage, that's the only thing preventing two agents, two PRs and twice the bill.
• needs-human sticks. When a worker gives up, it comments, adds the needs-human label and hands the card back. Without the label, the next poll picks it up again and comments again, every ten seconds.
• Two tokens. The workers need a broad token to read boards and move cards (GitHub Projects v2 only accepts classic tokens). The sandbox only needs to push one branch and open one PR, so it gets a narrow, fine-grained token. Assume the sandbox can leak, and give it the token you can afford to lose.
• Manual sync. The ArgoCD apps for the factory don't auto-sync. Every run costs real money, so nothing should restart it by accident. Tasks are never committed to the synced folder either, or ArgoCD would recreate them and rerun the agent forever.

What the first day in production taught me

Everything was green in unit tests. Then it met the board, and the first day found five real bugs. None of them were about the code; all of them were about the world.

"Retry later" is wrong for a decision

The worker loop assumed "this failed, try again later". That's right for a network blip and wrong for a decision. Three different failures were decisions retried forever, at model prices:

• Starvation. Checking whether a card is blocked costs an API call, so each poll checked at most 20 cards. But the scan always restarted at the top. With 20 blocked cards first in the column, the worker reported "everything is blocked" while 80 workable cards waited behind them. The fix is a moving window that continues where the last poll stopped.
• Truncated answers. When the model hit its token limit, it was treated as an error and retried, producing the same too-long answer. Now it's a decision: comment, label, hand back.

The rule I carry forward: when you add a failure path, ask whether a retry could plausibly give a different result. If not, it's a decision. Comment, label, return the card.

The timeout that wasn't

Plans simply never appeared. The errors said Client.Timeout exceeded and connection reset by peer, which read like a network problem. So I raised the timeout from 5 to 15 minutes. It didn't help.

The real cause: the API closes a response that has sent nothing for about 60 seconds. A non-streaming request sends nothing until it is done, so any plan that took more than a minute to write could never arrive. One curl showed it:

non-streaming  max_tokens=12000   reset after 61 s
streaming      max_tokens=12000   200 OK after 169 s, 4878 chunks

Switching to streaming fixed it. Two details matter:

• data: [DONE] is the completeness check. A stream that stops early looks exactly like a short answer. Without checking for the terminator, a half-written plan gets posted as if the model meant to stop there, and then an agent executes it.
• Ask for usage explicitly. With streaming, token usage only comes if you set stream_options.include_usage, and it arrives in a final chunk with no choices.

The agent that couldn't build

The first task failed with Agent produced no changes. The sandbox image had git, gh and curl, but no Node, and step one of the plan was npx .... The agent spent its whole run trying to install Node and gave up.

An agent that can't run the project's build can't know whether its change compiles. Every PR before that fix was a guess. The sandbox image now has Node, Go and Python, but no Docker, Helm or kubectl.

Prompts need limits

The plan prompt asked for "the specifics the implementer cannot guess" and never said how much. On a vague issue, the model wrote a 22,000-character plan. Now plans are capped at twelve steps, and if more are needed the model marks the card blocked and suggests splitting it. A plan that's too long isn't a thorough answer. It's no answer, because it arrives truncated.

Is it worth it?

For small, well-described changes, yes: a new page, a config change, copy edits, a dependency bump. I spend my time on the two approvals and the PR review, not on the typing. For big or vague work, the outline step is where it pays off: the factory tells me early that the issue isn't clear enough, which is useful on its own.

What I'd tell anyone building one:

1. Keep humans on the gates that matter, and enforce it in code, not by convention.
2. Make every failure path either a retry that can succeed or a decision that hands back.
3. Give the agent the tools to run the build.
4. Measure before you fix. One curl would have saved me a day.