Static analysis and the fuzzer in CI, and what the rules caught #82

2026-10-15T10:59:44Z

extant2000git commented

2026-10-15 10:59:44 +00:00

docs/BEST-PRACTICES.md declared eight criteria unmet for want of static and dynamic analysis, and the letter to an auditor points at that document. This closes them.

scripts/check-static.py is six rules aimed at what this engine can actually have. The one worth having alone is a WebCrypto Promise used as a value: if (SUBTLE.verify(...)) without await is truthy for a forged signature as readily as for a good one. The others are Math.random anywhere in src/, eval or new Function, loose equality on a secret-shaped name, an empty catch outside browser storage, and JSON.parse returned with no shape check.

The first version flagged nine things and seven were its own fault, which is how a rule teaches people to ignore it:

src/e2e.js:123: [await] importKey() without await      <- return SUBTLE.importKey(...)
src/e2e-client.js:503: [catch] empty catch block       <- around localStorage.setItem

Six were return SUBTLE.importKey(...) from an async function, which is correct and idiomatic. Two were empty catches around localStorage.setItem, which throws on a quota or in a private window and should be swallowed. Both rules are narrowed, and both narrowings are explained where someone will read them.

The ninth was real. openBackup returned JSON.parse of the decrypted plaintext with no shape check, so a backup whose payload was null or a bare string came back to a caller that reads fields off it. AES-GCM has authenticated those bytes under a passphrase-derived key, so it is a smaller hazard than the payload parser test/fuzz.mjs found, and it is the same bug. It throws backupPayloadNotAnObject now.

Every rule was checked by planting the bug it looks for:

src/e2e.js:46: [random] Math.random is not a CSPRNG
src/e2e.js:47: [await] verify() used as a value without await
src/e2e.js:48: [equality] loose equality near 'tag'
src/e2e.js:49: [catch] empty catch block

Then removing the plants and confirming the tree is clean.

test/fuzz.mjs joins the same analysis job, because a check nobody runs stops finding things. The CI comment states the measured runtime, 45 seconds, rather than the 11 I guessed first.

Gate. Static analysis clean across 10 files, 8,001 fuzz cases with 0 problems, conformance 304 passed with 2 skipped, client 289, ratchet 35, vault 26, all 0 failures. Secret sweep clean across 91 files, doc-references resolves 153 sections, and check-exports-documented passes 106 exports.

🤖 Generated with Claude Code

`docs/BEST-PRACTICES.md` declared eight criteria unmet for want of static and dynamic analysis, and the letter to an auditor points at that document. This closes them. `scripts/check-static.py` is six rules aimed at what this engine can actually have. The one worth having alone is a WebCrypto Promise used as a value: `if (SUBTLE.verify(...))` without `await` is truthy for a forged signature as readily as for a good one. The others are `Math.random` anywhere in `src/`, `eval` or `new Function`, loose equality on a secret-shaped name, an empty catch outside browser storage, and `JSON.parse` returned with no shape check. **The first version flagged nine things and seven were its own fault**, which is how a rule teaches people to ignore it: ``` src/e2e.js:123: [await] importKey() without await <- return SUBTLE.importKey(...) src/e2e-client.js:503: [catch] empty catch block <- around localStorage.setItem ``` Six were `return SUBTLE.importKey(...)` from an async function, which is correct and idiomatic. Two were empty catches around `localStorage.setItem`, which throws on a quota or in a private window and should be swallowed. Both rules are narrowed, and both narrowings are explained where someone will read them. **The ninth was real.** `openBackup` returned `JSON.parse` of the decrypted plaintext with no shape check, so a backup whose payload was `null` or a bare string came back to a caller that reads fields off it. AES-GCM has authenticated those bytes under a passphrase-derived key, so it is a smaller hazard than the payload parser `test/fuzz.mjs` found, and it is the same bug. It throws `backupPayloadNotAnObject` now. **Every rule was checked by planting the bug it looks for:** ``` src/e2e.js:46: [random] Math.random is not a CSPRNG src/e2e.js:47: [await] verify() used as a value without await src/e2e.js:48: [equality] loose equality near 'tag' src/e2e.js:49: [catch] empty catch block ``` Then removing the plants and confirming the tree is clean. `test/fuzz.mjs` joins the same `analysis` job, because a check nobody runs stops finding things. The CI comment states the measured runtime, 45 seconds, rather than the 11 I guessed first. **Gate.** Static analysis clean across 10 files, 8,001 fuzz cases with 0 problems, conformance 304 passed with 2 skipped, client 289, ratchet 35, vault 26, all 0 failures. Secret sweep clean across 91 files, `doc-references` resolves 153 sections, and `check-exports-documented` passes 106 exports. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

extant2000git added 1 commit 2026-10-15 10:59:45 +00:00

Static analysis and the fuzzer in CI, and what the rules caught

ci / secret-sweep (pull_request) Successful in 5s

Details

ci / ratchet (pull_request) Successful in 5s

Details

ci / vault (pull_request) Successful in 6s

Details

ci / conformance (pull_request) Successful in 9s

Details

ci / doc-references (pull_request) Successful in 3s

Details

ci / complete (pull_request) Successful in 2s

Details

ci / spec-counts (pull_request) Successful in 26s

Details

ci / client (pull_request) Successful in 33s

Details

ci / analysis (pull_request) Successful in 1m30s

Details

4e18eae131

docs/BEST-PRACTICES.md declared eight criteria unmet for want of static
and dynamic analysis, and the letter to an auditor points at that
document. This closes them.

scripts/check-static.py is six rules aimed at what this engine can
actually have. The one worth having alone is a WebCrypto Promise used as
a value: `if (SUBTLE.verify(...))` without `await` is truthy for a forged
signature as readily as for a good one. The others are Math.random
anywhere in src/, eval or new Function, loose equality on a
secret-shaped name, an empty catch outside browser storage, and
JSON.parse returned with no shape check.

🔴 The first version flagged nine things and seven were its own fault,
which is how a rule teaches people to ignore it. Six were
`return SUBTLE.importKey(...)` from an async function, which is correct
and idiomatic. Two were empty catches around `localStorage.setItem`,
which throws on a quota or in a private window and should be swallowed.
Both rules are narrowed and both narrowings are explained where someone
will read them.

The ninth was real. `openBackup` returned `JSON.parse` of the decrypted
plaintext with no shape check, so a backup whose payload was `null` or a
bare string came back to a caller that reads fields off it. AES-GCM has
authenticated those bytes under a passphrase-derived key, so it is a
smaller hazard than the payload parser that test/fuzz.mjs found, and it
is the same bug. It now throws `backupPayloadNotAnObject`.

Every rule was checked by planting the bug it looks for and confirming
it fires, then removing it and confirming the tree is clean.

test/fuzz.mjs joins the same `analysis` job. A check nobody runs stops
finding things, and this one found a real defect on the day it was
written. The CI comment states the measured runtime, 45 seconds, rather
than the 11 I guessed first.

Gate: static analysis clean across 10 files, 8,001 fuzz cases with 0
problems, conformance 304 passed with 2 skipped, client 289, ratchet 35,
vault 26, all 0 failures. Secret sweep clean across 91 files and
doc-references resolves all 153 sections.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

extant2000git merged commit 29882f0802 into main

2026-10-15 11:01:53 +00:00

extant2000git deleted branch ci/static-analysis

2026-10-15 11:01:53 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: extant2000/flamenet-e2e#82