AI · Agentic Systems
machinist
A GitHub-native multi-agent pipeline that turns a one-paragraph request into a spec’d, implemented, adversarially reviewed, tested pull request. A human approves the spec and merges the PR; everything in between is autonomous.
The idea
Agents are good at producing code and bad at judging their own output. machinist splits those concerns: agents propose — code, review findings, verdicts, prose — while a deterministic orchestrator disposes, deciding every stage transition through a pure, exhaustively tested state machine. The result is a pipeline autonomous enough to ship a pull request unattended, and accountable enough that every decision it made is readable in the issue thread afterwards.
How it works
- An interactive /spec flow turns a one-paragraph request into a validated spec — testable acceptance criteria, non-goals, a self-review checklist — filed as a GitHub issue with priority and effort labels.
- A human applies the approval label; that single label event (verified to come from a write-access user) triggers a GitHub Actions run that drives every remaining stage inside a non-root Docker sandbox.
- An implementer agent builds the change on a branch and opens a draft PR; four read-only specialist reviewers — correctness, security, tests, style — then critique the diff in parallel.
- An adversarial challenger tries to refute every finding they raise. Only confirmed blocker-level findings send the code back to a fixer; the loop escalates to a human after three strikes.
- The orchestrator itself runs the test suite — the gate is exit-code truth, never an agent's claim — and a scribe writes the final PR description before the draft flips to ready.
- Every control-flow decision lives in a pure, exhaustively unit-tested state machine over the issue's labels. Agents produce artifacts; they never decide what happens next.
The run that proves it
Issue #12 — a tagging feature for the demo API — ran unattended on GitHub Actions for just under nine minutes. The implementer built the feature; the review panel’s test specialist caught that the tag-filter test couldn’t distinguish exact matching from substring or case-insensitive matching — precisely the behaviors the spec ruled out — and the challenger confirmed it. Strike one. The fixer added near-miss fixtures to pin the behavior down. In round two, a security finding about unbounded tag counts was struck down by the challenger as out of spec. The test gate passed, and the scribe flipped PR #13 to ready with the whole honest history in its description — one real defect caught, one false positive killed, in the same run.
Decisions worth calling out
An agent whose only job is killing false positives
Reviewer findings are claims, not facts. The challenger must refute each one with proof or confirm it — and its silence counts as confirmation. In the pipeline's first fully autonomous run it did both jobs in one pass: confirmed a subtle real defect and struck down a plausible-sounding finding as out of spec.
GitHub is the database
Labels hold the state machine, issue threads hold the audit trail, structured comment blocks carry findings between stages, and Actions is the runtime. There is no orchestration server, no queue, no store to operate — and every pipeline decision is publicly inspectable after the fact.
Deterministic gates wherever possible
The test gate is the orchestrator running the suite and reading the exit code. Agents are reserved for judgment calls — what to build, whether a finding is real — while everything checkable by a machine is checked by a machine, including format, types, lint, and workflow hygiene on the pipeline's own repo.
Bounded autonomy
Nothing runs until a person applies the approval label, and the pipeline never merges its own PRs. Strike-limited fix loops, per-agent tool allowlists and turn caps, a job timeout, and a read-only sandbox mount bound what an unattended run can do.
What it demonstrates
Multi-agent orchestration where the orchestration itself is boring, testable code; adversarial review as a first-class mechanism rather than a prompt suggestion; and the discipline of deterministic gates around probabilistic workers. The pipeline was also debugged in production against the real world — headless-CLI deadlocks, Windows argv mangling, GitHub’s bot-identity restrictions — and each incident is documented in the repo’s history with a regression test attached.
Want to talk about building agentic systems like this?
Get in touch