Stop the secret sweep failing on a clean tree (relay#53)

`token: "deadbeef"` — an attachment id in a truncation test — trips the
"assigned password" pattern, and has on `main` for a while. I hit it
publishing relay#50 and had to stop and prove the match was pre-existing
before I could trust my own branch.

🔴 **A gate that fails on a clean tree is worse than no gate.** It teaches you
to run it and ignore the answer, and the one time it catches something real
the habit is already to shrug.

⚠️ **The two obvious fixes are both worse.** Narrowing the pattern until
`deadbeef` slips through puts a hole in the SHAPE — the next real secret that
happens to be lowercase goes with it. Allow-listing the value silences
`deadbeef` everywhere, including a file where it is real.

So: a per-line `// secret-sweep: fixture` marker, and nothing else. It must be
typed at the site by somebody who looked at the line, and it is greppable.

🔴 **The count is pinned, because a printed number is not a gate.** Nobody
reads the output of a passing check, so "suppressions are counted" would have
been decoration — somebody could mark a real credential as a fixture and this
would say `clean` in green. `EXPECTED_FIXTURES` makes adding a marker FAIL
until the number is bumped in the same change: a second, deliberate act, in a
diff a human is already reading.

Proved all four ways rather than just the first, which is what the ticket
asked for:

  clean tree                          → clean (190 files, 1 fixture)
  planted sk_live_ with no marker     → FAILED, Stripe live key
  planted sk_live_ WITH a marker      → FAILED, 2 markers, expected 1,
                                        and it prints the offending line
  plant removed                       → clean again

659 tests in 60 suites pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Paul Hitt
2026-09-07 07:48:06 -04:00
co-authored by Claude Opus 5
parent 13ffce9a4d
commit ffdbef88f6
2 changed files with 63 additions and 4 deletions
+60 -3
View File
@@ -37,13 +37,50 @@ if [ -z "$FILES" ]; then
FILES=$(find . -type f -not -path '*/.build/*' -not -path '*/node_modules/*' -not -path '*/.git/*')
fi
# 🔴 **The one way to silence a line, and it is deliberately annoying.**
#
# A gate that fails on a clean tree is worse than no gate: it teaches you to run
# it and ignore the answer, and the one time it catches something real the habit
# is already to shrug. This happened — `token: "deadbeef"`, an attachment id in
# a truncation test, is genuinely the shape of an assigned credential.
#
# ⚠️ **The two obvious fixes are both worse than this one.** Narrowing the
# pattern until `deadbeef` slips through puts a hole in the SHAPE, and the next
# real secret that happens to be lowercase goes with it. Allow-listing the value
# silences `deadbeef` everywhere, including the file where it is real.
#
# So: a line carrying the marker below is skipped, and nothing else is. It is
# per-line, it must be typed at the site by somebody who looked at the line, it
# is greppable, and the count is printed on every run so suppressions cannot
# grow quietly. A marker on a real credential is possible — that is true of any
# suppression — but it is a visible, reviewable, deliberate act rather than an
# accident.
MARKER='secret-sweep: fixture'
# 🔴 **Pinned, because a printed number is not a gate.** Nobody reads the output
# of a passing check, so "suppressions are counted" would have been decoration:
# somebody could mark a real credential as a fixture and this script would say
# `clean` in green. Pinning the count means adding a marker FAILS until the
# number here is bumped in the same change — a second, deliberate, reviewable
# act, in a diff a human is already looking at.
#
# ⚠️ Bumping this is not a formality. Read the line that made it necessary and
# satisfy yourself it is a fixture before you change the number.
EXPECTED_FIXTURES=1
hits=0
suppressed=0
probe() {
local label="$1" pattern="$2"
local found
found=$(printf '%s\n' "$FILES" \
local all found skipped
all=$(printf '%s\n' "$FILES" \
| grep -vE '(^|/)vendor/|\.build/|secret-sweep\.sh$' \
| xargs -I{} grep -HnE "$pattern" {} 2>/dev/null || true)
# Split rather than filter-and-forget: a suppression nobody counts is a
# suppression nobody notices growing.
found=$(printf '%s\n' "$all" | grep -vF "$MARKER" | grep -v '^$' || true)
skipped=$(printf '%s\n' "$all" | grep -cF "$MARKER" || true)
suppressed=$((suppressed + skipped))
if [ -n "$found" ]; then
echo "── $label"
printf '%s\n' "$found" | head -8
@@ -73,4 +110,24 @@ if [ "$hits" -gt 0 ]; then
echo "secret sweep FAILED: $hits pattern(s) matched. Nothing was published."
exit 1
fi
echo "secret sweep clean ($(printf '%s\n' "$FILES" | wc -l | tr -d ' ') files)"
if [ "$suppressed" -ne "$EXPECTED_FIXTURES" ]; then
echo
echo "secret sweep FAILED: $suppressed line(s) carry '$MARKER', expected $EXPECTED_FIXTURES."
if [ "$suppressed" -gt "$EXPECTED_FIXTURES" ]; then
printf '%s\n' "$FILES" \
| grep -vE '(^|/)vendor/|\.build/|secret-sweep\.sh$' \
| xargs -I{} grep -HnF "$MARKER" {} 2>/dev/null | head -8
echo
echo "Read each line above. If every one is genuinely a fixture and not a"
echo "credential, set EXPECTED_FIXTURES=$suppressed in this script."
else
echo "A marker was removed. Set EXPECTED_FIXTURES=$suppressed."
fi
exit 1
fi
note=""
if [ "$suppressed" -gt 0 ]; then
note=", $suppressed line(s) marked as fixtures"
fi
echo "secret sweep clean ($(printf '%s\n' "$FILES" | wc -l | tr -d ' ') files$note)"
@@ -52,7 +52,9 @@ final class AttachmentIntegrityTests: XCTestCase {
/// somebody looking at the client.
func testTheLogPointsAtStorageRatherThanAtTheClient() throws {
let message = try XCTUnwrap(AttachmentIntegrity.report(
token: "deadbeef", verdict: .truncated(expected: 1024, found: 512)))
// An attachment id, not a credential.
token: "deadbeef", // secret-sweep: fixture
verdict: .truncated(expected: 1024, found: 512)))
XCTAssertTrue(message.contains("deadbeef"), message)
XCTAssertTrue(message.contains("1024"), message)
XCTAssertTrue(message.contains("512"), message)