Stop the secret sweep failing on a clean tree (relay#53)
`token: "deadbeef"` — an attachment id in a truncation test — trips the "assigned password" pattern, and has on `main` for a while. I hit it publishing relay#50 and had to stop and prove the match was pre-existing before I could trust my own branch. 🔴 **A gate that fails on a clean tree is worse than no gate.** It teaches you to run it and ignore the answer, and the one time it catches something real the habit is already to shrug. ⚠️ **The two obvious fixes are both worse.** Narrowing the pattern until `deadbeef` slips through puts a hole in the SHAPE — the next real secret that happens to be lowercase goes with it. Allow-listing the value silences `deadbeef` everywhere, including a file where it is real. So: a per-line `// secret-sweep: fixture` marker, and nothing else. It must be typed at the site by somebody who looked at the line, and it is greppable. 🔴 **The count is pinned, because a printed number is not a gate.** Nobody reads the output of a passing check, so "suppressions are counted" would have been decoration — somebody could mark a real credential as a fixture and this would say `clean` in green. `EXPECTED_FIXTURES` makes adding a marker FAIL until the number is bumped in the same change: a second, deliberate act, in a diff a human is already reading. Proved all four ways rather than just the first, which is what the ticket asked for: clean tree → clean (190 files, 1 fixture) planted sk_live_ with no marker → FAILED, Stripe live key planted sk_live_ WITH a marker → FAILED, 2 markers, expected 1, and it prints the offending line plant removed → clean again 659 tests in 60 suites pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
13ffce9a4d
commit
ffdbef88f6
+60
-3
@@ -37,13 +37,50 @@ if [ -z "$FILES" ]; then
|
||||
FILES=$(find . -type f -not -path '*/.build/*' -not -path '*/node_modules/*' -not -path '*/.git/*')
|
||||
fi
|
||||
|
||||
# 🔴 **The one way to silence a line, and it is deliberately annoying.**
|
||||
#
|
||||
# A gate that fails on a clean tree is worse than no gate: it teaches you to run
|
||||
# it and ignore the answer, and the one time it catches something real the habit
|
||||
# is already to shrug. This happened — `token: "deadbeef"`, an attachment id in
|
||||
# a truncation test, is genuinely the shape of an assigned credential.
|
||||
#
|
||||
# ⚠️ **The two obvious fixes are both worse than this one.** Narrowing the
|
||||
# pattern until `deadbeef` slips through puts a hole in the SHAPE, and the next
|
||||
# real secret that happens to be lowercase goes with it. Allow-listing the value
|
||||
# silences `deadbeef` everywhere, including the file where it is real.
|
||||
#
|
||||
# So: a line carrying the marker below is skipped, and nothing else is. It is
|
||||
# per-line, it must be typed at the site by somebody who looked at the line, it
|
||||
# is greppable, and the count is printed on every run so suppressions cannot
|
||||
# grow quietly. A marker on a real credential is possible — that is true of any
|
||||
# suppression — but it is a visible, reviewable, deliberate act rather than an
|
||||
# accident.
|
||||
MARKER='secret-sweep: fixture'
|
||||
|
||||
# 🔴 **Pinned, because a printed number is not a gate.** Nobody reads the output
|
||||
# of a passing check, so "suppressions are counted" would have been decoration:
|
||||
# somebody could mark a real credential as a fixture and this script would say
|
||||
# `clean` in green. Pinning the count means adding a marker FAILS until the
|
||||
# number here is bumped in the same change — a second, deliberate, reviewable
|
||||
# act, in a diff a human is already looking at.
|
||||
#
|
||||
# ⚠️ Bumping this is not a formality. Read the line that made it necessary and
|
||||
# satisfy yourself it is a fixture before you change the number.
|
||||
EXPECTED_FIXTURES=1
|
||||
|
||||
hits=0
|
||||
suppressed=0
|
||||
probe() {
|
||||
local label="$1" pattern="$2"
|
||||
local found
|
||||
found=$(printf '%s\n' "$FILES" \
|
||||
local all found skipped
|
||||
all=$(printf '%s\n' "$FILES" \
|
||||
| grep -vE '(^|/)vendor/|\.build/|secret-sweep\.sh$' \
|
||||
| xargs -I{} grep -HnE "$pattern" {} 2>/dev/null || true)
|
||||
# Split rather than filter-and-forget: a suppression nobody counts is a
|
||||
# suppression nobody notices growing.
|
||||
found=$(printf '%s\n' "$all" | grep -vF "$MARKER" | grep -v '^$' || true)
|
||||
skipped=$(printf '%s\n' "$all" | grep -cF "$MARKER" || true)
|
||||
suppressed=$((suppressed + skipped))
|
||||
if [ -n "$found" ]; then
|
||||
echo "── $label"
|
||||
printf '%s\n' "$found" | head -8
|
||||
@@ -73,4 +110,24 @@ if [ "$hits" -gt 0 ]; then
|
||||
echo "secret sweep FAILED: $hits pattern(s) matched. Nothing was published."
|
||||
exit 1
|
||||
fi
|
||||
echo "secret sweep clean ($(printf '%s\n' "$FILES" | wc -l | tr -d ' ') files)"
|
||||
if [ "$suppressed" -ne "$EXPECTED_FIXTURES" ]; then
|
||||
echo
|
||||
echo "secret sweep FAILED: $suppressed line(s) carry '$MARKER', expected $EXPECTED_FIXTURES."
|
||||
if [ "$suppressed" -gt "$EXPECTED_FIXTURES" ]; then
|
||||
printf '%s\n' "$FILES" \
|
||||
| grep -vE '(^|/)vendor/|\.build/|secret-sweep\.sh$' \
|
||||
| xargs -I{} grep -HnF "$MARKER" {} 2>/dev/null | head -8
|
||||
echo
|
||||
echo "Read each line above. If every one is genuinely a fixture and not a"
|
||||
echo "credential, set EXPECTED_FIXTURES=$suppressed in this script."
|
||||
else
|
||||
echo "A marker was removed. Set EXPECTED_FIXTURES=$suppressed."
|
||||
fi
|
||||
exit 1
|
||||
fi
|
||||
|
||||
note=""
|
||||
if [ "$suppressed" -gt 0 ]; then
|
||||
note=", $suppressed line(s) marked as fixtures"
|
||||
fi
|
||||
echo "secret sweep clean ($(printf '%s\n' "$FILES" | wc -l | tr -d ' ') files$note)"
|
||||
|
||||
@@ -52,7 +52,9 @@ final class AttachmentIntegrityTests: XCTestCase {
|
||||
/// somebody looking at the client.
|
||||
func testTheLogPointsAtStorageRatherThanAtTheClient() throws {
|
||||
let message = try XCTUnwrap(AttachmentIntegrity.report(
|
||||
token: "deadbeef", verdict: .truncated(expected: 1024, found: 512)))
|
||||
// An attachment id, not a credential.
|
||||
token: "deadbeef", // secret-sweep: fixture
|
||||
verdict: .truncated(expected: 1024, found: 512)))
|
||||
XCTAssertTrue(message.contains("deadbeef"), message)
|
||||
XCTAssertTrue(message.contains("1024"), message)
|
||||
XCTAssertTrue(message.contains("512"), message)
|
||||
|
||||
Reference in New Issue
Block a user