← Guides · 日本語 · Português · 中文

Updated 2026-11-19

On this page
  1. The pattern
  2. When to use it
  3. Step 1 — Secure the reference first
  4. Step 2 — Decompose into owned slices
  5. Step 3 — Humans steer above the code
  6. Step 4 — Adversarial validation on top
  7. Verify it
  8. Pitfalls
  9. Knowledge check
  10. Sources

Guide 5 — Reference-grounded swarm

Goal: direct a swarm of agents to write a large, verifiable system — where correctness is checked continuously against a trusted reference, not against vibes. Level: advanced Source deployment: Harvey's "Phil" — ~30 agents wrote a collaborative, Word-compatible editor engine in Rust + TypeScript: 1M+ lines in 3 weeks, zero human engineers. Verified continuously against real Microsoft Word behavior and simulated lawyer negotiations. Humans steered through memory + decision logs, never by reading code. Sources: Gabe Pereyra's launch write-up (via @gabepereyra) · launch post

The pattern

A swarm fails when nobody can tell whether the output is right. Phil worked because every line could be checked against a reference of truth: real Microsoft Word. The agent team's job was matching — an objective, continuously checkable target — not inventing.

   reference system (truth)
            ▲ compare
            │
┌───────────┴────────────┐
│  swarm of ~30 agents   │──► 1M+ LOC (Rust + TypeScript)
└────────────────────────┘
            ▲ steer
            │
   humans: memory + decision logs
   (never reading the code)

When to use it

Good fit Bad fit
Reimplementing a spec (file format, protocol, editor, compiler) Greenfield products with no objective answer
Porting/migration with an oracle (old system still runs) Novel UX where "right" is a judgment call
Anything diff-able against a working system Systems where correctness needs users

The litmus test: can a machine decide "correct" without you? If yes, a swarm can scale; if no, use a product-owner Bot (Guide 2) instead.

Step 1 — Secure the reference first

Before any agent writes code, the check must exist:

Put the comparison in CI or a script the swarm runs constantly — "does output match reference" must be cheap and automatic, or agents drift between checks.

Step 2 — Decompose into owned slices

~30 agents didn't share one backlog — the work was sliced so each agent owned a subsystem (document model, layout engine, collab protocol, …). Structure it like Guide 4's org chart, except every "employee" is a coding session pointed at a module boundary:

You own the <module>. Its contract is <interface/reference behavior>.
Your PRs are accepted when the reference checks pass — not when the
code looks done.

Step 3 — Humans steer above the code

Nobody at Harvey read the million lines. Steering happened through:

Your review surface is the decision trail, not the diff. If you find yourself reading generated code to know what's happening, ask the orchestrator for the decision log instead.

Step 4 — Adversarial validation on top

Harvey layered a second check: simulated lawyer negotiations — realistic use that surfaces what a conformance suite misses. Your equivalent: a scenario layer above the reference diff — real workflows, adversarial inputs, fuzzing. Reference checks catch wrong; scenario checks catch brittle.

Verify it

Pitfalls

Knowledge check

  1. Why did Harvey's swarm work where most multi-agent attempts collapse? (A trusted external reference — real Word behavior — made correctness machine-checkable at swarm speed.)
  2. Why did humans steer through memory and decision logs instead of code review? (1M+ LOC in 3 weeks exceeds any human read capacity; steering happens at the intent layer.)
  3. What's the prerequisite you'd check before attempting this pattern? (Whether your domain has a reference the swarm can check itself against — if not, build the oracle first.)

Sources