Paul HittandClaude Opus 5
a9ec8e98bb
Seven of 52 UI tests failed in one run with "sign-up did not complete". The
app was right and the tests were wrong, and the failure dump said so:
signup.password value: • ← one character
signup.confirm value: ••••••••••••••••• ← all seventeen
signup.submit Disabled
iOS puts "Use Strong Password?" over the first `.newPassword` field, and from
the first keystroke on it swallows what follows. Every sign-up helper in this
target called `dismissPasswordSuggestionIfPresent()` — at the end, after both
passwords had been typed, by which point there is nothing left to save. The app
then correctly refused a form whose two passwords disagreed, and the report
blamed sign-up for it.
It is non-deterministic, which is why it reads as flake: a fifth suite running
identical keystrokes passed in the same run. It is not flake and it is not
machine load.
There were five hand-rolled copies of that helper — four here, plus one I wrote
for the new two-simulator suite, which reproduced the bug immediately and is
what makes the point. Now there is one, `XCUIApplication.completeSignUp`, and it
dismisses the panel when the field takes focus, reads both fields back, repairs a
short one, and fails naming the cause instead of the app.
Two new suites:
TwoSimulatorUITests — two simulators, two accounts, one relay. Both sides run
the same test with the handles swapped, so neither has to start first; the
peer lookup retries, which is the rendezvous. A direct message crosses in both
directions and a group created on one device arrives on the other. This is
weaker evidence than CrossPlatformUITests and says so in its own header: both
halves are the same binary and will agree when both are wrong. What it proves
that nothing else does is that a message crosses two devices, two Keychains
and two ratchets and lands in the app's own UI.
AppSurfaceUITests — the screens nothing was opening. Reactions, unsend,
in-chat search, mute, the voice-note recorder, the stories strip, the groups
section, the safety number and five of the seven settings screens had no test
at all. Deliberately shallow and it says so: this catches a screen that
crashes or fails to push, not one that is wrong.
Also here, each found while running the above:
- DEVELOPMENT_TEAM was "2W7DA3A8W2", the free personal team on the build Mac,
while the App Store provisioning profile that already exists for
net.flamenet.messenger belongs to M5923Y84P5 — the team that ships hittfitness
and shepherdgiving, valid to 2027-06-04. A personal team cannot produce a
distribution build, so every archive for TestFlight would have failed naming
the certificate rather than the team ID that put it out of reach.
- Scripts/generate.sh, because .gitignore excludes the whole .xcodeproj and
SwiftPM keeps Package.resolved inside it. Since the WebRTC dependency landed a
fresh checkout had no pins at all: resolve against the network and get a
different build tomorrow, or disable resolution and fail outright. Both
unacceptable for something going to testers. The pins live in Scripts/ now.
- rounds.sh ran an iOS *build* and called it a round. The 45 XCUITest cases
never ran. It runs them now — and refuses to score a run that executed zero
tests as a pass, which xcodebuild will happily report after a runner crashes
and relaunches:
Test Suite 'TwoSimulatorUITests' passed
Executed 0 tests, with 0 failures
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Flamenet Messenger — Android
Nothing here is an app yet. This is the engine, and it is deliberately first.
An Android client needs the protocol implemented a third time. That third
implementation is the one that finds the places where the specification says
less than the two existing engines quietly agreed on — but only if it is
written against the spec and the published vectors rather than against a
reading of our own Swift and JavaScript. So the engine comes first, it is a
plain Kotlin/JVM library with no Android in it at all, and it is held to
engine/test/vectors/protocol-v1.json — the same file the other two engines
are checked against, read from that one location rather than copied here.
./gradlew :engine:test
No Android SDK, no emulator, no device. It runs anywhere a JVM does, which is also what lets it run in CI on Linux.
What the engine covers today
| Area | Spec | State |
|---|---|---|
| Safety numbers | §7 | Done — fingerprints and conversation ordering match the published vectors |
| Key transparency verification | §19 | Done — RFC 6962 leaf/node hashing, unbalanced split, bottom-up audit paths |
| Byte helpers | — | Unsigned comparison, constant-time equality |
What it does not cover
Listed rather than left to be discovered, because a table of green ticks with the hard parts quietly missing is worse than no table:
| Area | Spec | State |
|---|---|---|
| X25519 / Ed25519 | §3 | Not started — needs a provider that works on both JVM and Android |
| X3DH key agreement | §4 | Not started |
| Double Ratchet | §6 | Not started |
| Sealed sender | §12 | Not started |
| ML-KEM-768 hybrid | §11 | Not started |
| ML-DSA-65 identities | §20 | Not started |
| Relay client | §8 | Not started |
| The app itself | — | Not started |
Until those exist, Android is honestly marked "not started" on the website, and a conversation gets the protocol's guarantees only when both ends are iOS.
Why the vectors matter more than the code
Two implementations that agree may both be wrong in the same way, especially when one was written while reading the other. The audit-path ordering in §19 is the worked example: a top-down reading of a bottom-up path produces the correct root for every left-spine index, so it passes any test that only checks index 0. The vector file pins a right-descending index for that reason, and the Kotlin tests assert both that the published path verifies and that the same path reversed does not.
If this engine ever disagrees with the file, the interesting question is which of the three is wrong — not this one by default.