Both agents were handed the identical prompt and the identical model — Claude Fable 5. The only variable is the harness: Amp Ultra (amp -x --mode ultra) versus Claude Code (claude -p --effort high). Amp built it faster and cheaper — but a game is meant to be played, so judge the result yourself.
We ran three cold runs per agent (benchmark strategy below) and link only the best run of each here, so you compare each harness at its strongest. Games open in a new tab. Controls: WASD move · mouse look · left-click fire · right-click pump · R reload · Esc pause. Click the game to lock the pointer.
Selected as Amp’s best: cheapest and largest output of its three runs, delivered in under 7 minutes.
▶ Play the Amp gameSelected as Claude’s best: its fastest and cheapest run with the most code. *All Claude runs ended in an API error, but the game HTML was recoverable.
▶ Play the Claude gameBoth agents used Claude Fable 5, so these gaps reflect the harness, not the model.
| Run | Amp duration | Amp cost | Amp LOC | Claude duration | Claude cost | Claude LOC |
|---|---|---|---|---|---|---|
| Cold 1 | 397.1 s | $2.93 | 1,118 | 765.4 s | $4.80 | 1,253 |
| Cold 2 | 547.6 s | $2.60 | 1,150 | 1,107.9 s | $8.84 | 1,170 |
| Cold 3 | 416.5 s | $2.42 | 1,218 | 986.6 s | $7.20 | 1,196 |
| Average | 453.8 s | $2.65 | 1,162 | 953.3 s | $6.95 | 1,206 |
| 3-run total | $7.95 | $20.84 — exhausted the Claude Max monthly limit | ||||
Efficiency: Amp 2.56 LOC/s and $2.28 per 1,000 LOC; Claude 1.27 LOC/s and $5.76 per 1,000 LOC. Output size was similar (~1,100–1,200 lines) — more lines is not inherently a better game. Amp cost is zero-markup pass-through, read via amp threads usage.
The quantum benchmark on the previous page compares different models in different harnesses, so a skeptic can argue the model is doing the work. This page cannot be argued away: both agents ran Claude Fable 5 on the verbatim prompt, so model capability is held constant and the 2.1× / 2.6× gap is pure harness.
matches the 2026 academic definition
A May 2026 survey, Code as Agent Harness (arXiv:2605.18747, UIUC × Meta × Stanford), names its open problem #1 as “harness-level evaluation and oracle adequacy” — the complaint that public benchmarks “conflate the capabilities of the base model, the quality of the harness, the reliability of tools, the informativeness of feedback, and the difficulty of the environment.” This experiment is designed to not conflate: the model and the prompt are pinned, so only the harness can vary.
That is why this page asks you to play the games, not read the code. The paper calls this an execution-based harness representation — the verdict is the runtime, not a linguistic judgment. Four of the survey’s six harness-level metric dimensions are directly instantiated here:
In the paper’s vocabulary, this is the experiment that isolates the harness as the independent variable and uses execution as the oracle. Every other public benchmark the survey cites (SWE-bench, OSWorld, MLE-bench…) varies the model alongside the harness and then reports a single conflated score. The baboons face-off is the controlled-trial version of the same idea — and it is the one a skeptic cannot dismiss as “maybe the other model was just better.”
Agentic coding is stochastic — one run can flatter or flunk either agent. So each agent got three independent cold runs: fresh context every time, no reference implementation, no reusing a previous attempt (no “cold-start” carry-over). Averaging across runs measures the harness’s typical behaviour rather than a lucky sample. On this page we then surface the single best run of each so you can play each harness at its strongest — the full six runs are in the repo.
Nothing was tailored per agent. The full specification — weapon ballistics, spread formula, five levels, London environment, HUD, audio — was handed to both, verbatim: