Engineering Software LabEngineering Software LabCloning, hardening, optimizing, testing & documenting a real public Terraform testing library — with rigorous, hallucination-free verification of every change.
Twenty-four slides across six chapters — from the initial ChatGPT brief through security hardening, a real bug fix, measured performance gains, rigorous verification, and the two upstream PRs. Tap any line to jump to that slide.
“I need a public Terraform demo project that will be used to show the capabilities of Ampcode to improve the performance of a Terraform project and test the improved code — with rigorous testing to make sure the conversion was done right: no bugs, no AI hallucinations.”
Apache-2.0 · maintained by Gruntwork.io
Terratest is a library imported by your _test.go files: it deploys real infrastructure, asserts against it, then tears it down. Scale measured from the clone:
Required toolchain (auto-upgraded during the work).
Permissive OSS license · maintained by Gruntwork.
The v1 line receives security fixes only; v2 is in development. This shaped our PR choices.
Supporting trees: examples/ (sample infra) · test/ (integration) · cmd/ (CLI) · internal/
Guardrail: every claim must be backed by a reproducible command — no unverified assertions, no hallucinated fixes.
Shallow clone of Terratest
443 files · 22 modules mapped
Grype + Trivy · Claude Mythos
Regexp hoisting · global rand
UniqueID collisions
build · vet · test · bench · re-scan
Report + scorecard
Submitted upstream
Severity distribution (Trivy, go.mod) — with 3 rated CRITICAL by Grype among the x/crypto set.
Also flagged: actions/download-artifact@v4 (CI supply chain, High) and docs-site Ruby gems (out of product scope).
Terratest's modules/ssh imports x/crypto/ssh directly — this isn't a dormant transitive dependency, it's on the library's real attack surface.
| Dependency | From | To | Resolves | |
|---|---|---|---|---|
| golang.org/x/crypto | 0.49.0 | → | 0.52.0 | 14 SSH CVEs (Critical/High) |
| golang.org/x/net | 0.52.0 | → | 0.55.0 | HTML/HTTP2 XSS & DoS |
| jackc/pgx/v5 | 5.9.0 | → | 5.9.2 | SQL injection (database) |
| ulikunitz/xz | 0.5.10 | → | 0.5.15 | Memory-leak DoS |
| golang.org/x/sys | 0.42.0 | → | 0.45.0 | Overflow + required by x/crypto |
Discovered when TestUniqueID failed — not caused by our edits.
Coarse Windows timer resolution makes the collision reliable; finer Linux CI timers hid it.
Technique: hoist constant regexp.MustCompile out of per-call scope into package-level vars (compile once); replace per-call PRNG allocation with the global source. Log scale ↓
Pre-existing Windows-only test failures (needing Unix cat / diff) were identified and clearly labelled as not caused by our changes — no green-washing.
| Claim | Reproducible evidence |
|---|---|
| “CVEs are fixed” | trivy fs go.mod → 0 · grype dir:. → 0 |
| “It's faster” | go test -bench -benchmem (before vs after) |
| “The bug is fixed” | go test -count=5 -run TestUniqueID → PASS |
| “Nothing broke” | go build ./... & go vet ./... → PASS |
| “Changes are minimal” | git diff — 7 files, isolated per PR |
If it couldn't be shown with a command, it wasn't claimed.
Quality > quantity — don't flood the maintainers.
fix(random): prevent UniqueID collisions from per-call time seeding
Correctness bug fix · isolated to modules/random.
github.com/gruntwork-io/terratest/pull/1860 →fix(deps): bump dependencies to resolve known CVEs
Security · go.mod/go.sum only · go mod tidy-clean.
github.com/gruntwork-io/terratest/pull/1861 →Submitted from fork zuwasi/terratest against gruntwork-io/terratest:main. A CLA signature may be required before merge.
AI-assisted engineering, done responsibly: measured, verified, and contributed back — with zero hallucinations.