QuickSWE v2
C++ & CUDA Benchmark

Amp Deep³
cross-provider reasoning
vs
Claude Opus 4.7
Anthropic model tested April 2026

April 17, 2026  |  41 Valid Tasks  |  WSL2 + CUDA 12.8 + g++ 13.3

22 C++ tasks (easy → extreme)  •  19 CUDA tasks (easy → extreme)

Update — July 2026: This page reports an April 17, 2026 QuickSWE v2 run using Amp Deep³ and Claude Opus 4.7. Newer experiments test Amp Ultra/Rush and Claude Fable 5 with different tasks and methodology, so their scores are not directly comparable to the 97.6% vs 82.9% results shown here. See Baboons Benchmark · Harness Beats the Model · Final Scores

← Swipe to navigate →

Executive Summary

97.6%
Amp Deep³ Resolve Rate
82.9%
Claude Opus 4.7 Resolve Rate
40/41
Amp Resolved
34/41
Claude Resolved
1
Amp Regressions
3
Claude Regressions
Amp Deep³ resolves 18% more tasks
and causes 3× fewer regressions than Claude Opus 4.7

⚙️ C++ Results — 22 Valid Tasks

100%
Amp Deep³ (22/22)
90.9%
Claude Opus 4.7 (20/22)

By Difficulty

🟢 Easy (5 tasks)Both 100%
Amp 5/5
Claude 5/5
🟡 Medium (4 tasks)Both 100%
Amp 4/4
Claude 4/4
🟠 Hard (8 tasks)Amp leads
Amp 8/8
Claude 7/8
🔴 Extreme (5 tasks)Amp leads
Amp 5/5
Claude 4/5

🟢 CUDA Results — 19 Valid Tasks

94.7%
Amp Deep³ (18/19)
73.7%
Claude Opus 4.7 (14/19)

By Difficulty

🟢 Easy (4 tasks)Both 100%
Amp 4/4
Claude 4/4
🟡 Medium (5 tasks)Amp leads
Amp 5/5
Claude 4/5
🟠 Hard (4 tasks)Amp leads
Amp 3/4
Claude 2/4
🔴 Extreme (6 tasks)Amp dominates
Amp 6/6
Claude 4/6

📈 Difficulty Scaling

How each agent degrades as task difficulty increases

Difficulty
Amp Deep³
Claude Opus 4.7
🟢 Easy (9)
100% (9/9)
100% (9/9)
🟡 Medium (9)
100% (9/9)
88.9% (8/9)
🟠 Hard (12)
91.7% (11/12)
75.0% (9/12)
🔴 Extreme (11)
100% (11/11)
72.7% (8/11)
Claude drops from 100% on easy tasks to 72.7% on extreme
Amp stays at 91.7–100% across all levels — virtually flat

⚙️ Head-to-Head: C++ Tasks

TaskDifficultyDescriptionAmpTimeClaudeTime
051EasyCircular buffer wrap-around✅77.9s✅22.1s
052EasyString tokenizer escapes✅73.3s✅59.6s
053EasyMatrix mul dimension indexing✅29.4s✅11.2s
054EasyMin-heap sift-down✅48.3s✅15.4s
055EasyHash map linear probing✅62.4s✅32.6s
056MedSFINAE type dispatch✅86.3s✅39.7s
057MedShared_ptr ref counting✅77.2s✅11.1s
059MedIterator invalidation in erase✅47.0s✅13.6s
060MedVariadic fold expression✅62.1s✅40.4s
061HardRed-black tree insertion fix-up✅41.0s✅19.2s
062HardB+ tree key redistribution✅53.8s✅105.7s
063HardPool allocator free-list✅38.9s✅36.3s
065HardPatricia trie split logic✅100.0s✅189.1s
066HardTarjan's SCC lowlink✅120.1s❌11.3s
067HardPratt parser precedence✅101.0s✅41.9s
068HardNFA→DFA epsilon closure✅46.7s✅25.5s
069ExtMark-compact GC forwarding ptrs✅123.7s✅84.2s
071ExtB-tree lazy deletion rebalance✅122.4s✅339.2s
072ExtCoroutine scheduler transfer✅175.6s❌ ⏰600.1s
073ExtConstexpr ray tracer reflect✅40.8s✅17.1s
074ExtSIMD matrix alignment/masks✅145.0s✅⚠️112.9s
075ExtHAMT persistent data structure✅88.4s✅23.2s

⏰ = timeout (600s)   ⚠️ = regression (broke existing tests)

🟢 Head-to-Head: CUDA Tasks

TaskDifficultyDescriptionAmpTimeClaudeTime
076EasyVector add grid/block dims✅42.9s✅17.6s
078EasyHistogram atomicAdd races✅43.7s✅12.4s
079EasyPrefix sum offset error✅41.4s✅14.0s
080Easy1D convolution halo cells✅106.2s✅34.6s
081MedWarp shuffle reduction mask✅54.1s❌14.1s
082MedCSR SpMV row pointer indexing✅62.7s✅14.8s
083MedBitonic sort non-power-of-2✅71.6s✅254.9s
084MedStream compaction scatter✅75.9s✅66.3s
085MedRadix sort signed integers✅53.3s✅21.1s
087HardCooperative groups tile partition✅71.7s❌24.8s
088HardUnified memory prefetch hints✅42.5s✅34.8s
090HardWarp-divergent predication✅88.5s✅72.3s
091HardTensor core WMMA layout❌⚠️54.4s❌⚠️18.8s
094ExtDynamic parallelism streams✅89.4s❌⚠️58.3s
095ExtCustom GEMM tiling + double buf✅66.7s✅71.3s
096ExtBFS warp-centric frontier✅109.5s❌36.8s
097ExtFFT Cooley-Tukey twiddle✅81.4s✅240.5s
099ExtRay tracing BVH traversal✅46.2s✅37.3s
100ExtMolecular dynamics neighbor list✅66.4s✅65.2s

⚠️ = regression (broke existing tests)

🔍 Where Claude Opus 4.7 Failed

7 tasks failed — failures cluster on GPU-specific concepts

task_066 Hard C++
Tarjan's SCC algorithm — incorrect lowlink update (11.3s — gave up fast)
task_072 Extreme C++
Custom coroutine scheduler — incorrect symmetric transfer (⏰ TIMEOUT 600s)
task_081 Medium CUDA
Warp-level shuffle reduction — incorrect mask (14.1s — gave up fast)
task_087 Hard CUDA
Cooperative groups tile partition — incorrect tile size for partial warps
task_091 Hard CUDA
Tensor core WMMA fragment layout — row vs col major mismatch (both agents failed)
task_094 Extreme CUDA
Dynamic parallelism — nested kernel launch stream management + regression
task_096 Extreme CUDA
BFS graph traversal — warp-centric frontier expansion
Pattern: Claude struggles with GPU warp semantics, tensor cores, and cooperative groups
5 of 7 failures involve CUDA-specific parallel primitives

⚠️ Regression Analysis

Regressions = agent broke existing passing tests while trying to fix bugs

1
Amp Regressions
3
Claude Regressions

Amp Deep³ — 1 Regression

• task_091 (CUDA Hard) — Tensor core WMMA layout. Failed to resolve AND regressed.

Claude Opus 4.7 — 3 Regressions

• task_074 (C++ Extreme) — SIMD matrix alignment. Resolved but broke pass_to_pass tests.
• task_091 (CUDA Hard) — Tensor core WMMA layout. Failed AND regressed.
• task_094 (CUDA Extreme) — Dynamic parallelism streams. Failed AND regressed.

Claude Opus 4.7 is 3× more likely to break what already works

🏆 Verdict

Amp Deep³ wins decisively

97.6%
Amp Overall
82.9%
Claude Overall
100% vs 90.9%
C++ Resolve Rate
94.7% vs 73.7%
CUDA Resolve Rate

Claude Opus 4.7 struggles with GPU-specific concepts:
warp shuffle masks • cooperative groups • tensor cores • dynamic parallelism

Amp Deep³ maintains near-perfect accuracy across all difficulty levels
100% on easy through extreme C++ • 94.7% on CUDA

✅ Zero safety violations from either agent (ACL-protected directories)