Learning from other frameworks and platforms
Who else is solving this, what I took from them, and when to use them instead of, or alongside, this playbook.
Checked November 2, 2026. This field changes every month. Re-check links and status before you rely on them.
Many people are working out how to build software with fleets of coding agents. Some of them have far more users than this playbook. Others have stronger evidence. This playbook is not the only answer, and it is not meant to be. It is the working system from my own work: the team layer of queue, budget, human decisions and records. It sits on top of methods and tools that others built.
This section does three things:
- It lists what I have adopted from other people, and how far along each adoption is.
- It maps the wider landscape, so you can pick the right tool for your situation.
- For each entry, it says when to use it instead of this playbook, and how to combine the two.
The related-work list in Agentic Kanban covers team-process proposals: AWS AI-DLC, OpenAI's AI-native team guide, Gas Town, Böckeler's review of spec-driven tools, IBM, SDAD, Agentsway and others. This section does not repeat those. It covers methods, platforms, memory, review and measurement.
Status labels used below:
| Label | Meaning |
|---|---|
running |
Part of my daily system today |
piloting |
Being tried on real work; results not in yet |
considering |
Looks useful; not tried or decided |
reference |
Worth reading; nothing to adopt directly |
What I adopted, and from whom
| From | What I took | Status | Where it shows up |
|---|---|---|---|
| Claude Code (Anthropic) | Headless claude -p workers and reviewers, dollar caps per headless session (--max-budget-usd with -p), a fresh-context reviewer for every PR, hooks for rules that must never break |
running |
Execute, Review, Guardrails |
| Superpowers (Jesse Vincent) | Brainstorm → plan → test-first → review discipline inside each session | running |
Agentic Kanban: the per-task method under the team layer |
| Codex (OpenAI) | A second model family for deep review, so the reviewer is not the same model that wrote the code | running |
Review |
| Anthropic's best practices | "Rules that must never break go in hooks, not prose": my cost guard blocks one-path-per-page CDN invalidations after an incident | running |
Guardrails |
Claude Code /code-review |
A second, independent reviewer run on a sample of PRs that my deep review already passed, to measure what my review misses | piloting |
Review, Measure |
| HumanLayer research → plan → implement | For large work, a human reads a short plan before workers start. The research gets neutral questions about how the system works today, kept apart from the desired change | considering |
Execute, Epic lane |
| HumanLayer design-control-loop | Standing loops: a sensor measures a gap, a controller picks one small fix, one PR at a time, with reviewer feedback carried between runs. My drift checks already exist as sensors; the loop around them does not | considering |
Learn, Workflows |
| OpenAI harness engineering | When review raises the same finding a third time, encode it as a lint rule or structural test instead of another comment. Keep the agent instruction file short and use it as a map into deeper docs | considering |
Review, Guardrails |
| calldiff | A call-tree diff generated by a tool, not by the author agent. Tells a tester which paths changed and tells a reviewer which callers are affected | considering |
Review |
| tdd-guard | A hook that blocks implementation edits until a failing test exists. Turns "test first" from a prose rule into a mechanical one | considering |
Guardrails |
| HumanLayer's test-output hook | Block piping test or typecheck output through head, tail or grep, so a failure cannot be filtered out of sight. I lost 30 minutes once to a silent filtered poll loop |
considering |
Guardrails |
| METR, DORA | Measurement method: a baseline, confidence intervals, stability and rework next to throughput | considering |
Measure, evidence |
Quick chooser
| Your situation | Start with | Add this playbook when |
|---|---|---|
| One developer, one task at a time, wants discipline | Superpowers | You run several tasks at once and need a queue, budgets and a record of decisions |
| A large feature with open product questions | Spec Kit or OpenSpec | The spec turns into many tasks that run in parallel. The spec becomes the brief's Spec field |
| Your organization runs Scrum and wants familiar roles | BMAD Method or AWS AI-DLC | You want to drop the meetings and batch human decisions one to one |
| A solo greenfield prototype with strong tests | The Ralph loop | Teammates, reviews or production data enter the picture |
| You drive 3–5 agents by hand and watch them | Conductor or claude-squad | Work must run while nobody watches |
| You want an open, self-hosted multi-agent runner | Gas Town | You need cost per accepted task, budget caps and decision records |
| You want hosted delegation with no runner to maintain | GitHub Copilot coding agent, Devin or Factory | You want the brief format and decision records on top: the issue body becomes the brief |
| Memory for one repo or one developer | AGENTS.md or Cline Memory Bank | Decisions and sessions span people and repos |
Methods: how one task gets done
This playbook is a team layer. It assumes some per-task method underneath it, and these are the main ones.
Superpowers (Jesse Vincent). Skills for Claude Code that fire automatically: brainstorm, plan, write the test first, run subagents, review. running in my sessions.
- Use it for discipline inside every task.
- Combine: Superpowers runs inside each worker session. The playbook decides which task runs, with what budget, and who answers its questions.
GitHub Spec Kit and OpenSpec. Spec-driven development: a written spec is the source of truth and code is generated from it. Spec Kit is heavier. OpenSpec keeps each change in its own folder and is built for existing codebases (Böckeler). - Use them for large features where the product shape is still open. - Combine: the plan they produce becomes the Spec field of the task brief. Keep the short brief for small and medium tasks.
BMAD Method. Persona agents for analyst, product manager, architect, scrum master, developer and QA. - Use it when the organization wants agile roles and artifacts kept. This playbook replaces roles with lanes and decision records.
HumanLayer: research → plan → implement (Dex Horthy). Keep the context window 40–60% full. Split work into research, plan and implementation, each in a fresh session. A human reviews the plan, not the code: "I can't read 2000 lines of golang daily. But I can read 200 lines of a well-written implementation plan." The method is packaged as the rpi skill. Two ideas from it are worth taking even if you don't use the skill:
- Neutral research. The research agent gets only questions about how the system works today. The desired change is kept in a separate document, so the research isn't bent toward one answer.
- Research describes; review judges. The research step reports what exists, where it lives and how it works. Judging the code is the reviewer's job.
Dex Horthy later archived a 30,000–40,000-line codebase from a lightly supervised "software factory" experiment, and argues that verification, not generation, is the real bottleneck. That is the same lesson my Review chapter draws.
12-Factor Agents (HumanLayer). Design principles for agents: own your prompts and context window, contact humans through tool calls, keep agents small. My runner already follows several of them. Parking a task with a need: kind is "contact humans with tool calls", and the lean worker card is "own your context window". Read it as a checklist before you build or use a runner (Build or use).
Ralph loop (Geoffrey Huntley). A shell loop that feeds the same prompt file to an agent again and again, with state kept in files and git, and tests pushing back on bad output. - Use it for a solo greenfield prototype. It has no human decision layer or budget, which is fine when nothing shared is at risk.
Compound Engineering (Every) and Agent OS (Brian Casel). "Each unit of engineering work should make subsequent units easier." This matches the Learn section. reference.
Simon Willison: Agentic Engineering Patterns. The best plain-language practitioner guide I know: red/green test-first, "first run the tests", agentic manual testing, linear walkthroughs of agent changes, and anti-patterns such as filing unreviewed PRs. Recommended pre-reading before a pilot.
Kent Beck: Augmented Coding. Keep caring about the code, keep test-first, and watch for the agent deleting or weakening tests. Read it next to the "tests weakened" check in Review.
Gene Kim and Steve Yegge, Vibe Coding. The leadership framing: agents change how teams are organized, not only how fast code is written. A good read for executives before they read this playbook.
OpenAI: Harness engineering. OpenAI's account of building with agents writing the code. Two practices stand out: - The agent instruction file is a short map into a deeper docs folder, not a long rulebook. - Architecture rules are enforced by custom linters and structural tests, so agents can't drift from them.
It is ahead of this playbook on mechanical enforcement. Combine: when the same review finding comes back a third time, file a task to turn it into a rule.
Cognition: Coding Agents 101. Practical delegation advice: say how the result will be verified, start small, give the agent CI. Useful for choosing pilot tasks.
Anthropic: Claude Code best practices and How Anthropic teams use Claude Code. Hooks for hard rules, a short CLAUDE.md, team procedures as skills, a fresh-context reviewer and budget caps. I follow most of it. The gap I still have: some of my repo instruction files are far longer than this guidance recommends.
Factory Agent Readiness. Scores a repository on nine technical pillars and five maturity levels: fast tests, types, docs, reproducible environments and so on. Agents fail in repos that lack them. Combine: use the scores as the rows of the table that Assess produces.
HumanLayer skills: show-me and visual-pr. These explain a change with small text sketches: pseudocode, call-tree diffs, file trees and component trees. visual-pr writes a PR description in that style. It helps human reviewers. For agents, a tool-generated outline (calldiff, below) is safer, because the reviewer is not anchored on the author's own summary.
Running many agents
Gas Town and Beads (Steve Yegge). These have named roles: a coordinator, workers with short-lived sessions, a merge queue that bisects failures, monitors that detect stuck workers, and a version-controlled issue ledger (Beads, now backed by Dolt). It is the nearest open relative of the runner behind this playbook; see Agentic Kanban. - Use it when you want an open, self-hosted runner. - Use this playbook when you need budget caps, decision records and cost per accepted task. - Ideas worth borrowing: the bisecting merge queue, and detecting stuck workers by lack of progress rather than by spend.
Interactive tools for watched work: - Conductor: a Mac app for parallel agents in git worktrees. - claude-squad: tmux plus worktrees. - Sculptor (Imbue): agents in containers. Choose it when isolation matters more than speed.
These cover the "Parallel" adoption level, where a developer drives several agents by hand. The runner in this playbook covers unattended queues. Many teams need both.
Vibe Kanban (Bloop). A kanban board that dispatches cards to coding agents. It is the closest in name to Agentic Kanban. It has no budgets, decision records or acceptance gates. Bloop shut down in April 2026; the project continues as community-maintained open source. Fine as a free local board for one developer.
Claude Flow / Ruflo. "Swarm" orchestration with many agent types. It gets a lot of attention, but I found no independent data on outcomes or cost. Pilot it yourself before trusting it with production work.
OpenHands and opencode. Open, model-agnostic agent platforms. Use them if you don't want to depend on one model vendor. The process in this playbook (brief, gates, cost records) can sit on top of either.
Hosted agents. GitHub Copilot coding agent and Agent HQ, Devin and Factory take a ticket and return a PR, with no infrastructure to run. Cognition reports that 67% of Devin's PRs now merge; that is the vendor's own figure. Combine: write the issue as the six-field brief and keep the decision records. The hosted agent replaces the worker, not the team layer.
HumanLayer. Began as an SDK that routes an agent's tool calls to a human for approval, so the agent asks through the same channel it uses for everything else; the repository now hosts its agent-coordination app. The idea matches the way this playbook parks a task on a human decision.
A caution: Terragon was a hosted background-agent service that shut down in January 2026 and released an open-source snapshot. Hosted services come and go. Keep your task records, decisions and costs in a store you control (Build or use).
Memory and shared context
This playbook keeps team memory (tasks, decisions, sessions, guidelines) in a typed shared store. I use Personize, the product my company builds. You don't have to. Here is what else exists, with an honest view of when each is the better choice.
| Option | What it is | Use it when | Combine with this playbook |
|---|---|---|---|
| AGENTS.md | Open convention for repository instructions, read by most coding agents | Always, for per-repo rules | Keep it short, as a map. Put team records in the shared store |
| Cline Memory Bank | Six markdown files the agent keeps up to date | One developer, no shared service | A good starting point at the "Assisted" level |
| Beads | Version-controlled issue and dependency ledger for agents | A team that won't adopt a hosted memory service | Can hold the task ledger; decisions still need a home |
| mem0, Letta, Graphiti | Open agent-memory libraries with large communities | You'd rather build than buy | Check them against the eight requirements in Build or use |
| Augment Context Engine | Semantic code retrieval over MCP | Large codebases where agents miss related code | Code retrieval is a different need from team memory; you may want both |
| CodeRabbit learnings, Greptile memory | Review tools that remember feedback | You use those reviewers | They read your existing CLAUDE.md files; keep the rules in one place |
Personize's case rests on typed records, governance and safe concurrent updates across people and repositories. It is not the only memory option, and the libraries above are open and widely used.
Review and verification
Review is the bottleneck in this playbook, and in most of the work above. HumanLayer, OpenAI and StrongDM all treat verification as what limits how far you can scale.
| Tool | What it adds | Status here |
|---|---|---|
Claude Code Review and local /code-review |
Several agents look for problems in parallel, then a verification step removes false positives. Never approves or blocks | piloting as a second reviewer. First result is below the table |
| claude-code-action, claude-code-security-review | Reviews and security passes in CI. The security action is not hardened against prompt injection; use it on trusted PRs only | considering for the capped security audits in Workflows |
| CodeRabbit, Greptile | Hosted PR reviewers with memory | reference. Judge them by false positives on your own PRs, not by vendor benchmarks |
| Graphite | Stacked PRs and AI review | reference. Stacks suit agent output; my median PR already shrank from 788 to 139 lines |
| calldiff | A call-tree diff between two commits, generated by a tool | considering for tester and reviewer prompts |
| tdd-guard | Blocks implementation edits until a failing test exists | considering |
First result (November 2, 2026). I ran /code-review high on 27 merged code PRs from October 29 to November 1 that my deep review had passed. Each was 31 to 839 changed lines. A second model checked every new major finding against the code. It rejected none, which says it agrees too easily, so I hand-checked five and re-rated all 18 using context it lacked, such as flags that are off in production.
8 of the 27 PRs (about 30%) still had a real defect of major severity or worse that the PR itself caused and my review had not mentioned. Depending on how you count paths behind flags that are off, the range is 8 to 12. Most were major rather than blocking.
The misses followed a pattern: - the fix covered one path, but the same data reached another; - a field was read under a different name than the one it was written under; - an error branch or early return left state behind; - a flag, once turned on, would meet existing data the change did not expect.
The second pass cost $0.96 per PR on average, about twice my deep review ($0.44); the checking model added $0.38. The next step is to add those patterns to the review prompt as explicit questions, then re-run the same 27 PRs as a replay set to see whether the miss rate drops.
A limit they all share: most AI reviewers, including mine, read code. They don't run it. Running the affected tests in review helps. Behaviour scenarios, as in StrongDM's approach, go further.
Measuring: benchmarks and team measurement
Two different questions often get mixed up:
- Model benchmarks ask how capable a model is. They help you choose a model. They say nothing about your team.
- Team measurement asks whether your team ships more good software for the money and time. This is what the playbook's numbers try to answer.
Model benchmarks (context only)
| Benchmark | What it measures | Why it matters here |
|---|---|---|
| SWE-bench Pro | Real issues from 41 repositories, built to resist training-data leakage | The right outside reference when talking about model capability |
| Terminal-Bench, run on Harbor | Terminal tasks; a Harbor task is an instruction, an environment and a test script | Its task format is a good template for a private replay set of your own past tasks |
| SlopCodeBench | How code erodes as specs change over many checkpoints | The closest match to agents making long chains of changes in one codebase, which is how this playbook works |
| METR time horizon | The length of task, in human-expert time, at which a model is predicted to succeed half the time | Explains why task size limits matter |
Team measurement (the bar to meet)
| Source | What it shows | What to take from it |
|---|---|---|
| METR developer uplift | Randomized trials. In early 2025, tasks took 19% longer with AI. The late-2025 update estimates a speedup, but METR calls its data "very weak evidence", and time records were unreliable for developers running several agents at once | Randomize where you can. Plan how you will log human time before you claim savings |
| DORA 2025 | Nearly 5,000 respondents. AI magnifies a team's existing strengths and weaknesses. It is now linked to higher delivery throughput; making sure software works before it ships remains the challenge | Report stability (change failure rate, rework) next to throughput |
| DX AI Measurement Framework | Utilization, impact and cost | Its cost dimension matches cost per accepted task |
| SPACE | No single metric stands for productivity | Keep several numbers |
| Faros AI, 2025 and 2026 | Large telemetry studies. In 2025, merged PRs rose 98% with no company-level gain. In 2026, PRs were bigger, with more bugs and incidents per PR | The strongest public warning against PR-count multipliers |
| GitClear | Duplicated code rising alongside AI assistance (a correlation) | Track duplication and erosion over time, not only per PR |
| ccusage | Token and cost usage per day and per session, from local agent logs | A starting point. Cost per accepted task also counts failed attempts and coordination |
What this means for the numbers in this playbook
The playbook's measured results are records from my own work over a few months, and the playbook says so. "20×" is my goal, not a measured multiplier. Measured against the sources above, the honest gaps are:
- No baseline or control. METR randomizes tasks. A matched period before agents, or a small randomized set, would make claims stronger.
- Human time is not measured. Without it, a multiplier has no denominator.
- Stability is thin. Reverts, follow-up fixes and incidents within 30 days of merge belong next to throughput, as DORA reports them.
- PR counts mislead. Faros shows they can rise with no real gain. Cost per accepted task, with failures included, is the better lead number.
- "Accepted" is decided by the author's own system. Someone independent should check a sample.
- Runner and model changes are compared across different tasks. A frozen replay set of 20–30 past tasks with known-good outcomes would turn these comparisons into a repeatable benchmark.
I am working on these. The second-reviewer result above is a first step on gap 5. It also shows why a merge, even after review, is not acceptance: about 30% of the sampled PRs that passed my review still had a real defect the PR introduced.
Where this playbook fits
What this playbook adds (evidence is in my own records):
- Cost per accepted task, counting failed attempts and coordination. The coordinator was 49.6% of spend from October 10 to 13.
- Decisions treated as the queue. Most held work waited on a human decision, not on an agent. Of 1,109 questions agents asked me, I took the recommended answer 76% of the time, and only about 6–7% truly needed me (the study).
- Budget caps per worker, and unknowns recorded as unknown rather than guessed.
- Decision records that later sessions and reviewers read.
Where others are clearly ahead:
- Adoption and packaging. Superpowers, Spec Kit and Gas Town can be installed in minutes. My runner is not public yet.
- Measurement rigor. METR's trials and the DORA and Faros samples are far larger and better controlled than my own records.
- Mechanical enforcement. OpenAI's lint-and-structural-test approach goes further than my review prose.
- Repository readiness scoring. Factory has a ready-made rubric.
- Hosted review. Claude Code Review and CodeRabbit run with no setup.
How to combine them. Pick a per-task method (Superpowers, or Spec Kit for large features). Pick a worker harness (Claude Code, or OpenHands for model independence). Add a hosted reviewer if you have the budget. Then use this playbook for what sits above all of them: what runs next, with what budget, who decides, and what it cost per accepted result.