Same-model coding-agent benchmark

Amp Ultra vs Claude Code

Three independent cold runs per agent on the same assignment: “Create a first-person shooter game (London Baboon Hunter) with pump-action shotgun, 5 levels, Three.js.” Both agents used Claude Fable 5, isolating framework efficiency rather than model capability.

Same model: Claude Fable 53 cold runs per agentAmp Ultra · latest CLIClaude Code 2.1.209 · high effort

Executive summary

Six successful game outputs compared

Duration advantage
2.10×
Amp faster on average
Cost advantage
2.62×
Amp cheaper per run
Amp average
453.75s
$2.65 · 1,162 LOC
Claude average
953.31s
$6.95 · 1,206 LOC

Key findings

  1. Same model, different efficiency: Amp's agent framework was 2.1× faster and 2.6× cheaper.
  2. Output scale was similar: roughly 1,100–1,200 lines of functional Three.js game code.
  3. Amp achieved 2.56 LOC/s versus Claude's 1.27 and cost $2.28 versus $5.76 per 1,000 LOC.
  4. Amp's zero-markup, pass-through pricing reflects actual provider charges.
  5. All six runs produced games with the requested features; Claude's HTML remained recoverable despite all three sessions ending in API errors.
Bottom line: with Claude Fable 5 held constant, Amp delivered comparable-sized implementations with substantially lower elapsed time and cost.

Benchmark facts

  • Amp total: $7.95
  • Claude total: $20.84
  • Amp workspace balance moved from $14.66 to $18.87 because auto-replenishment occurred.
  • Claude's three sessions exhausted the Claude Max monthly spend limit.
  • Amp costs came from amp threads usage <thread-id>.

Performance charts

Hover or tap for exact values

Duration per run · seconds

Cost per run · USD

Lines of code per run

Average comparison · separate axes

Duration distribution · all 6 runs

Efficiency metrics

Per-run data

Measured output and actual cost

Amp Ultra · Claude Fable 5

Latest CLI, --mode ultra; Three.js r0.160.0 via unpkg.

RunDurationLOCFile sizeCostThread IDResult
Cold 1397.13s1,11844,790 B$2.93T-019f604e-8db1-7683-bd31-e3a1f468c8b2Game produced
Cold 2547.59s1,15046,589 B$2.60T-019f6054-aa58-705b-b213-34099a7276e8Game produced
Cold 3416.53s1,21844,561 B$2.42T-019f605c-ff48-746b-9291-e60608f499b9Game produced
Average453.75s1,162.045,313 B$2.653 / 3

Claude Code 2.1.209 · Claude Fable 5

High effort, Claude Max; minor Haiku auxiliary usage of approximately $0.0014 per run.

RunDurationLOCFile sizeCostInput tokensOutput tokensTerminal
Cold 1765.41s1,25350,591 B$4.8098,97863,885api_error
Cold 21,107.92s1,17048,781 B$8.84411,991126,291api_error
Cold 3986.61s1,19649,276 B$7.20291,086103,647api_error
Average953.31s1,206.3349,549 B$6.95267,35297,9410 / 3 clean exits

Head-to-head averages

MetricAmp UltraClaude CodeWinnerRatio / difference
Duration453.75s953.31sAmp2.10× faster
Cost per run$2.65$6.95Amp2.62× cheaper
Lines of code1,1621,206Claude3.7% more
File size45,313 B49,549 BSimilar~1×
LOC / second2.561.27Amp2.02×
Cost / 1,000 LOC$2.28$5.76Amp2.53×
Cost / second$0.0058$0.0073Amp1.25×
Total 3-run cost$7.95$20.84Amp2.62× cheaper

Technology comparison

Identical core stack

LanguageJavaScript ES modules
FrameworkThree.js r0.160.0 via CDN import map
ControlsPointerLockControls addons
RenderingWebGL
AudioWeb Audio API / AudioContext
PackagingSingle HTML; no dependencies beyond Three.js CDN

Amp used unpkg. Claude used jsDelivr in runs 1 and 3 and unpkg in run 2. These delivery choices do not change the application stack.

Notes & provenance

  • Model parity: both agents use Claude Fable 5. Amp Ultra is identified as “The most capable mode for hard, open-ended tasks — Claude Fable 5” by the Amp models page.
  • Amp pricing: zero-markup, pass-through API pricing; actual per-thread USD cost was retrieved with amp threads usage. See ampcode.com/pricing.
  • Version variance: installed Claude Code was 2.1.209, while the user-specified target was 2.1.206.
  • Claude recovery: file writes were denied in --print mode, so game HTML was recovered from permission_denials entries in result JSON rather than normal files.
  • Terminal status: all three Claude sessions reported terminal_reason: api_error and is_error: true, although complete game HTML was recoverable.
  • Spend limit: the three Claude runs cost $20.84 total, exhausting the Claude Max monthly spend limit.
  • Interpretation: output quantity is not a direct quality measure. This report shows measured speed, cost, and output scale; it does not claim that more lines inherently mean a better game.