RESEARCH BRIEF · 2026

Amp-Optimized Security Prompt

A multi-agent benchmark on the ArduPilot autopilot codebase

3 agents 2 prompts 6 runs 29 k files audited

Amp · Claude Code · Factory Droid
main_prompt.txt vs amp_security_prompt.txt

← Swipe / arrow keys to navigate →

THE STARTING POINT

`anshug/claude-mythos`

A community red-teaming prompt framework that turns an LLM into a multi-agent offensive security system.

Six specialized agents

Recon — maps attack surface
Hunter — finds vulnerabilities
Adversarial — chains exploits
Exploit — validates with PoCs
Triage — scores severity (CVSS)
AI Security — detects LLM-specific risks

Schema (original)

{
  "agent":        "HUNTER",
  "file_path":    "...",
  "vuln_class":   "...",
  "confidence":   "plausible",
  "cvss_vector":  "CVSS:3.1/...",
  "cvss_score":   9.6,
  "summary":      "...",
  "detail":       "# markdown..."
}

8 fields. No taint path. No per-metric CVSS rationale. No FP-disproof. Writes to /tmp/.

THE FIX

`amp_security_prompt.txt`

Same six-agent skeleton, but engineered for Amp's tool harness and against speculation.

Amp-native

Explicit guidance on Task subagents, finder, oracle, parallel reads
Tool-call budget enforced
Streaming [FOUND] / [DROP] status
Anti-prompt-injection guard

Anti-speculation

Mandatory taint path source → propagator → sink (file:line)
Per-metric CVSS justification (AV/AC/PR/UI/S/C/I/A)
why_not_false_positive with cited bypass
Confidence capped at plausible if no runtime

Auditable

Adversarial phase kills findings → rejected.jsonl
Stable finding_id from canonical sink signature
Mandatory scope.json + coverage-gaps section
.security/ (Windows-safe), not /tmp/

~25-field mandatory schema · AI-Security phase gated on has_llm_or_agent_code · Coverage-gaps section required

HOW WE TESTED

3 agents × 2 prompts = 6 runs on the same target

⚙️

Amp

🤖

Claude Code

🛠️

Factory Droid

×

📜

main_prompt.txt

⚡

amp_security_prompt.txt

↓

🎯

ArduPilot 4.6.3

→

📊

findings.jsonl × 6

→

🏆

Composite score

Controls

Same OS, same workspace, no internet, no runtime.

Budget

~80 tool-call cap per run; confidence capped at plausible.

Outputs

findings.jsonl + rejected.jsonl + scope.json

Scoring

8 weighted dimensions, 0–10 each, composite Σ(score × weight).

THE TARGET

ArduPilot Plane 4.6.3

Open-source autopilot used in real flying aircraft, rovers, submarines, and antenna trackers worldwide.

29 k

total files

2.7 k

C/C++ libraries

82%

C++ · 10% Python · 5% Lua

∞

attack surface

Attack-surface inventory (recon output)

Network entry points

MAVLink (UDP / UART / USB)
DDS XRCE-DDS over UDP:2019 (ROS 2)
SERIAL_CONTROL passthrough
AP_Networking TCP/UDP listeners
CAN / DroneCAN

Code-execution surface

Lua scripting (autoload from APM/scripts/)
MAVFTP arbitrary file write
SECURE_COMMAND signed firmware
Parameter store (EEPROM)

METHODOLOGY

8 weighted dimensions, scored 0 – 10

#	Dimension	Weight	Rationale
D1	Critical findings discovered	25 %	primary security value
D2	False-positive resistance	15 %	signal-to-noise
D3	Evidence quality (taint paths, CVSS rationale, disproof)	20 %	auditability
D4	Reproducibility (stable IDs, scope.json, artifacts)	10 %	re-runnable
D5	Coverage breadth (distinct vulnerable areas)	10 %	completeness
D6	Efficiency (findings per tool call)	5 %	cost
D7	Output-path portability (Windows + Linux)	5 %	operational
D8	Calibration / honesty (per-metric CVSS, coverage gaps)	10 %	anti-hallucination
	Total	100 %

RESULTS

Scoring matrix — all six runs

Rank	Run	D1	D2	D3	D4	D5	D6	D7	D8	Composite
🥇	Amp × amp_security	9	10	10	10	9	5	10	9	9.30
🥈	Claude × amp_security	7	10	10	10	8	7	9	10	8.85
🥉	Droid × amp_security	4	10	10	10	7	7	10	10	8.05
4	Droid × main_prompt	5	5	6	3	9	4	3	4	5.15
5	Claude × main_prompt	6	5	3	3	7	9	3	4	4.85
6	Amp × main_prompt	6	4	5	3	5	5	3	4	4.70

Two-factor effects

Prompt effect · +3.83 composite avg (range +2.90 to +4.60) — large

Agent effect · ±0.25 at composite — negligible

AT A GLANCE

Composite score per run

🥇 Amp × amp

9.30

🥈 Claude × amp

8.85

🥉 Droid × amp

8.05

Droid × main

5.15

Claude × main

4.85

Amp × main

4.70

The worst optimized-prompt run beats the best original-prompt run by +2.90 points.

THE WINNER

👑

Best composite score

9.30

Amp × `amp_security_prompt.txt`

6 Critical findings · 0 false positives · 8/8 taint paths · full artifact trail

+3.83

avg composite gain from the optimized prompt (across all 3 agents)

±0.25

agent variance — prompt structure dominates agent choice

100 %

of optimized-prompt runs produced taint paths and per-metric CVSS

IMPLICATIONS FOR ARDUPILOT

Six paths to LAN-adjacent vehicle takeover

Unauthenticated DDS · UDP 2019

ARM / DISARM service
MODE_SWITCH service
TAKEOFF service
SET_PARAMETERS · disables fence / arming-check
Joy topic · overrides all RC channels

CVSS up to 10.0 · AP_DDS_Client.cpp

MAVLink subsystem

MAVFTP file r/w + .. traversal
Drop .lua in APM/scripts/ → RCE
SERIAL_CONTROL → any UART
@SYS/storage.bin leaks signing key

CVSS 9.6 · GCS_FTP.cpp · GCS_serial_control.cpp

Signed-firmware fail-open

check_signature() returns true when keystore is zero
SETUP_SIGNING accepted pre-provisioning

CVSS 9.1 · AP_CheckFirmware_secure_command.cpp:199

Memory safety cluster

handle_device_op_write · 127-byte OOB read
@SYS lseek signed/unsigned OOB
Param upload OOB + biased writes
apfs_fgets int→uint8 truncation

CVSS 4–7 · surfaced via main_prompt.txt

WHAT WE PLAN TO FIX

Ordered by leverage

1 · Mandatory MAVLink signing

Gate FTP, SERIAL_CONTROL, SECURE_COMMAND, PARAM_SET behind signing_enabled(). Default GCS_SYSID_ENFORCE=1.

2 · DDS-Security profile

Auth + ACL + encryption on UDP 2019. Until then, bind DDS to 127.0.0.1 and require a WireGuard / IPsec tunnel.

3 · Close `check_signature` fail-open

Drop the all_zero_keys → true branch. Provisioning happens only over USB console.

4 · Allowlist MAVFTP paths

Realpath-normalize request.data; reject anything outside /APM, /@SYS, /@PARAM, /@MISSION, /@ROMFS.

5 · Sign Lua scripts

Refuse unsigned .lua from APM/scripts/ when written via a non-USB transport this boot.

6 · Memory-safety patches

Fix handle_device_op_write, @SYS lseek, apfs_fgets truncation, param-upload OOB. Add tests.

Items 1–3 alone collapse every Critical finding in this benchmark.

DON'T BOTHER · WONTFIX

Real risks · match ArduPilot's stated design

Upstream threat model: MAVLink and DDS run on operator-trusted links. Link auth and isolation are the operator's job, not the firmware's. PRs that bolt on auth get closed.

MAVFTP unauth + `..` traversal

Use SETUP_SIGNING and GCS_SYSID_ENFORCE=1. Path normalization breaks @SYS / @PARAM scripts.

DDS ARM / MODE / TAKEOFF / SET_PARAMS

DDS is for trusted LAN + companion. Use the DDS-Security profile. Per-packet auth is a non-starter.

SERIAL_CONTROL UART passthrough

Intentional — flashes GPS, talks to ESCs, debugs bootloaders. Always-on gating breaks every GCS workflow.

DDS Joy topic overrides RC

RC_Channels::set_override is canonical; the Joy topic is the documented ROS 2 interface. Removing it breaks Nav2 / Gazebo.

What CAN get merged · opt-in hardening parameters

FTP_OPTIONS bitmask · FTP_REQUIRE_SIGNING, FTP_ALLOWLIST_PATHS
DDS_AUTH_MODE · reject ARM / MODE / TAKEOFF unless origin authenticated
SERIAL_PASS_REQUIRE_SIGNING bit

Default behavior unchanged. Security-conscious operators get a switch to flip.

Strategy · ship 7 memory-safety / fail-open patches as bug fixes, then RFC PRs for opt-in hardening.

PULL REQUEST · UPSTREAM

ArduPilot/ardupilot#33147

Three atomic, non-breaking commits targeting master. Each is a pure bug fix preserving default behavior.

#	Subsystem	Class	File	+/−	Commit
1	`AP_CheckFirmware`	CWE-305	AP_CheckFirmware_secure_command.cpp	+13/−2	544689c4
2	`GCS_MAVLink`	CWE-125	GCS_DeviceOp.cpp	+7/−0	5efb802c
3	`AP_Filesystem`	CWE-125	AP_Filesystem_Sys.cpp	+25/−2	fb4410db

Total diff

+45 / −4 across 3 files. No new params, no API changes.

Atomic commits

Cherry-pickable. Maintainers can accept any subset.

Build verified

waf configure --board sitl && waf plane — clean build.

FIX #1 · CWE-305 · SIGNED-FIRMWARE FAIL-OPEN

`check_signature()` · bootstrap-only carve-out

libraries/AP_CheckFirmware/AP_CheckFirmware_secure_command.cpp

Before

if (all_zero_keys(sec_data)) {
    // allow through if no keys
    return true;   // ◀ FAIL-OPEN
}
if (pkt.sig_length != 64) {
    return false;
}

Every op passes when keys are zero — including GET_SESSION_KEY and REMOVE_PUBLIC_KEYS.

After

if (all_zero_keys(sec_data)) {
    // only allow bootstrap op
    return pkt.operation ==
        SECURE_COMMAND_SET_PUBLIC_KEYS;
}
if (pkt.sig_length != 64) {
    return false;
}

First-time provisioning still works. Every other op requires a valid Ed25519 signature.

Threat closed. Against an AP_SIGNED_FIRMWARE board that was never provisioned (or wiped via REMOVE_PUBLIC_KEYS), an attacker can no longer install keys, leak the session key, or flash signed bootloader payloads.

FIX #2 · CWE-125 · DEVICE_OP_WRITE OOB READ

`handle_device_op_write()` · count bound check

libraries/GCS_MAVLink/GCS_DeviceOp.cpp

Before · read handler had it, write did not

// READ path — bounded
if (packet.count > sizeof(data)) {
    retcode = 5; goto fail;
}

// WRITE path — unbounded
for (uint8_t i=0; i<packet.count; i++) {
    dev->write_bank_register(
        packet.bank,
        packet.regstart+i,
        packet.data[i]);  // ◀ OOB
}

packet.count up to 255; packet.data is 128 B. Stack leaks onto I2C/SPI.

After

// WRITE path
if (!dev) {
    retcode = 2; goto fail;
}
if (packet.count >
    sizeof(packet.data)) {
    retcode = 5;
    goto fail;   // ◀ added
}
// ... rest unchanged ...

Mirrors the read-handler check. 7 lines added, 0 removed.

Threat closed. A malicious or buggy GCS can no longer push 127 bytes of stack contents onto a hardware bus via DEVICE_OP_WRITE.

FIX #3 · CWE-125 · @SYS LSEEK WILD OOB READ

`AP_Filesystem_Sys::lseek()` · signed/unsigned safety

libraries/AP_Filesystem/AP_Filesystem_Sys.cpp

Before

case SEEK_SET:
    r.file_ofs = MIN(offset,
        (int32_t)length);
    break;
    // ◀ -1 → 0xFFFFFFFF
case SEEK_CUR:
    r.file_ofs = MIN(length,
        offset + r.file_ofs);
    break;
    // ◀ same hazard

// read() then does:
memcpy(buf,
    get_string()+file_ofs, count);
    // ◀ wild address

After

case SEEK_SET:
    if (offset < 0) {
        errno = EINVAL;
        return -1;
    }
    r.file_ofs = MIN(
        (uint32_t)offset, length);
    break;
case SEEK_CUR: {
    const int64_t new_ofs =
        (int64_t)r.file_ofs +
        (int64_t)offset;
    if (new_ofs < 0) {
        errno = EINVAL;
        return -1;
    }
    r.file_ofs = MIN(
        (uint32_t)new_ofs, length);
    break;
}

Threat closed. Any MAVFTP client could OpenFileRO /@SYS/storage.bin and lseek to a negative offset; the buffer base is live EEPROM or 0x08000000 flash — leaking MAVLink2 signing keys. Negative offsets now return EINVAL.

WORK WITH ESL

Need a security audit you can defend?

ESL — Engineering Software Lab — runs benchmarks like this for real customers: medical devices, automotive ECUs, robotics, aerospace, embedded firmware. We deliver auditable findings with taint paths, calibrated CVSS, and disproof evidence — not pattern-matched speculation.

Threat-led audits

Multi-agent offensive reviews of your codebase, with reproducible artifacts.

SBOM & supply chain

ESL SBOMator — component-level CVE intelligence with KEV/ransomware enrichment.

Standards & compliance

MISRA, CERT, AUTOSAR, IEC 62443, ISO 26262, FDA pre-market.

Amp-Optimized Security Prompt

anshug/claude-mythos

Six specialized agents

Schema (original)

amp_security_prompt.txt

Amp-native

Anti-speculation

Auditable

3 agents × 2 prompts = 6 runs on the same target

Controls

Budget

Outputs

Scoring

ArduPilot Plane 4.6.3

Attack-surface inventory (recon output)

8 weighted dimensions, scored 0 – 10

Scoring matrix — all six runs

Two-factor effects

Composite score per run

Amp × amp_security_prompt.txt

+3.83

±0.25

100 %

Six paths to LAN-adjacent vehicle takeover

Unauthenticated DDS · UDP 2019

MAVLink subsystem

Signed-firmware fail-open

Memory safety cluster

Ordered by leverage

1 · Mandatory MAVLink signing

2 · DDS-Security profile

3 · Close check_signature fail-open

4 · Allowlist MAVFTP paths

5 · Sign Lua scripts

6 · Memory-safety patches

Real risks · match ArduPilot's stated design

MAVFTP unauth + .. traversal

DDS ARM / MODE / TAKEOFF / SET_PARAMS

SERIAL_CONTROL UART passthrough

DDS Joy topic overrides RC

What CAN get merged · opt-in hardening parameters

ArduPilot/ardupilot#33147

Total diff

Atomic commits

Build verified

check_signature() · bootstrap-only carve-out

Before

After

handle_device_op_write() · count bound check

Before · read handler had it, write did not

After

AP_Filesystem_Sys::lseek() · signed/unsigned safety

Before

After

Need a security audit you can defend?

Threat-led audits

SBOM & supply chain

Standards & compliance

`anshug/claude-mythos`

`amp_security_prompt.txt`

Amp × `amp_security_prompt.txt`

3 · Close `check_signature` fail-open

MAVFTP unauth + `..` traversal

`check_signature()` · bootstrap-only carve-out

`handle_device_op_write()` · count bound check

`AP_Filesystem_Sys::lseek()` · signed/unsigned safety