A live suite cannot be pointed at production any more #33

2026-08-31T18:38:54Z

paulhitt commented

2026-08-31 18:38:54 +00:00

(Migrated from gitlab.com)

39 accounts were created in the production database in half an hour on
2026-08-31. 55 of the 86 accounts on that relay are now test accounts, which made
the launch signup metric unusable — a real person signing up was invisible inside
the burst.

Every one of those suites is correctly written: opt-in, with headers saying to
point them at the disposable fleet and not at production. That was the entire
protection, and a comment cannot refuse anything.

LiveTarget.refuseProduction is checked by every live suite before it does
anything, and throws rather than skips — "this run was pointed at production"
is not an unmet precondition, it is a mistake that has already done damage once,
and a skip is how it goes unnoticed the next time.

The guard-s own test found a bug in the guard

The disposable fleet lives on the same domain as production. Denying
flamenet.io and its subdomains refused relay1.flamenet.io — the exact place
these suites are meant to point:

LiveTargetTest > the disposable fleet and local relays are allowed() FAILED

It would have blocked every legitimate run while appearing to block only
production. The fleet is now named explicitly and checked first, which keeps the
bare domain denied so a production host added later is refused by default.

A deny list rather than an allow list, deliberately: an allow list would also
refuse a developer-s localhost or a self-hoster-s own relay, and a guard that
gets in the way is one somebody turns off.

Proven both directions

FN_LIVE_FLEET=https://flamenetmessenger.com
  -> IllegalStateException: refusing to run a live suite against flamenetmessenger.com

FN_LIVE_FLEET unset
  -> tests=1 skipped=1 failures=0

The guard-s own test runs in every pipeline, unlike the suites it protects —
those skip without FN_LIVE_FLEET, so a rule exercised only there would never be
checked at all.

Part of paulhitt/flamenet-relay#9. 39 accounts were created in the **production** database in half an hour on 2026-08-31. 55 of the 86 accounts on that relay are now test accounts, which made the launch signup metric unusable — a real person signing up was invisible inside the burst. Every one of those suites is correctly written: opt-in, with headers saying to point them at the disposable fleet and not at production. **That was the entire protection, and a comment cannot refuse anything.** `LiveTarget.refuseProduction` is checked by every live suite before it does anything, and **throws rather than skips** — "this run was pointed at production" is not an unmet precondition, it is a mistake that has already done damage once, and a skip is how it goes unnoticed the next time. ### The guard-s own test found a bug in the guard The disposable fleet lives on the **same domain** as production. Denying `flamenet.io` and its subdomains refused `relay1.flamenet.io` — the exact place these suites are meant to point: ``` LiveTargetTest > the disposable fleet and local relays are allowed() FAILED ``` It would have blocked every legitimate run while appearing to block only production. The fleet is now named explicitly and checked first, which keeps the bare domain denied so a production host added later is refused by default. A deny list rather than an allow list, deliberately: an allow list would also refuse a developer-s localhost or a self-hoster-s own relay, and a guard that gets in the way is one somebody turns off. ### Proven both directions ``` FN_LIVE_FLEET=https://flamenetmessenger.com -> IllegalStateException: refusing to run a live suite against flamenetmessenger.com FN_LIVE_FLEET unset -> tests=1 skipped=1 failures=0 ``` The guard-s own test runs in **every** pipeline, unlike the suites it protects — those skip without `FN_LIVE_FLEET`, so a rule exercised only there would never be checked at all.

paulhitt commented

2026-08-31 18:47:51 +00:00

(Migrated from gitlab.com)

mentioned in commit 7d823747df

mentioned in commit 7d823747df2d7c65023491caac8395f584d8b83f

paulhitt (Migrated from gitlab.com) merged commit 7d823747df into main

2026-08-31 18:47:51 +00:00

paulhitt commented

2026-09-09 19:01:57 +00:00

(Migrated from gitlab.com)

changed the description

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: extant2000/flamenet-messenger-android#33