Eighteen tests reported PASSED while running nothing #32

Merged
paulhitt merged 1 commits from 4-announce-skips into main 2026-08-31 17:56:04 +00:00
1 Commits
Author SHA1 Message Date
Paul Hitt 5242a506b3 Eighteen tests reported PASSED while running nothing
`val url = base ?: return` from an unmet precondition makes a test that did
nothing report as a pass. FN_LIVE_FLEET is never set in CI, so eighteen tests
across five files did this on every pipeline: LiveKeyLogTest 7,
LiveConversationTest 6, LiveRelayTest 2, LiveSealedSenderTest 2, LiveSignUpTest 1.

🔴 This repository had already written the rule down. LiveShapesTest uses
`assumeTrue` and says why, in its own words: a suite of tests that did nothing is
indistinguishable from a suite that works. Two files followed that and five did
not, so this is a consistency fix rather than a new idea.

It nearly reached the specification. `LiveSealedSenderTest` was the evidence §15
would have cited for "Android sealed sender, wired end to end" — two tests, zero
skipped, zero failures, on a run that did nothing. Only reading the source shows
that, and a status table is exactly where nobody would.

⚠️ The suite now looks worse and is not. Client goes from "277 tests, 0 failures"
to "277 tests, 28 skipped" — the 28 were always doing nothing; the number was
wrong before, not now. 578 of 606 actually run.
2026-08-31 13:54:39 -04:00