d4469aba163ef985ccbfafe5fea75eddaf991150
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul HittandClaude Opus 5
|
2764145063 |
Expose the security lattice to the interop driver (relay#49)
relay#49 requires that labels and security levels be kept as Messenger and Files converge. Both engines have implemented §1's lattice independently since the Files apps shipped, each pins its own vectors, and nothing has ever compared the two. 🔴 **The ten-byte encoding is authenticated data for every construction that uses a label**, so a platform disagreement does not warn. It produces a decryption failure, or — worse — plaintext that quietly belongs to the wrong sensitivity. Each side passing its own vectors is necessary and not sufficient; two engines can pass their own and still disagree. `label_encode` reports the mask as a STRING. Bit 63 makes it a negative Long, and a JSON number would arrive on the other side as a double and lose the low bits — turning the one compartment most likely to be mis-shifted into the one this check cannot see. The matching harness stage is flamenet-e2e!28. Proven by breaking it rather than by watching it pass: dropping `compartments.containsAll(...)` from `dominates` — the classic "compares the level and forgets the areas" bug — turned exactly four checks red, all of them compartment cases, each reporting iOS and Android disagreeing. That bug behaves correctly on every case built from levels alone, which is why it needs two engines to catch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
8f474ce0b9 |
Give the Kotlin engine content commands, so the layer can be checked (e2e#4)
The interop driver dispatched 19 commands — device, initiate, encrypt, seal, log_leaf — and not one touched a content envelope. Swift has had `content_encode`/`content_parse` for a while; Android has never been checked against any other implementation at this layer. 🔴 **This is the half that most needed it.** SPEC.md §6.10 says so in as many words, after the `ctl:true` bug: "no vector covers this envelope. It is the standing argument for the interop harness." That bug was Android writing `"ctl":true` where iOS decoded a number — and the cost was not a missed flag but a failed decode that dropped the whole message onto the plaintext path, while each engine's own tests passed on both sides. ⚠️ The two parsers fail DIFFERENTLY, which is why a round-trip on either proves so little. This engine reads field by field through `org.json`, whose bundled Android build coerces a JSON null to the string "null" where Maven's returns "". iOS decodes all-or-nothing through `Codable`, so one bad field downgrades the entire envelope to a raw-JSON bubble. The same additive change degrades differently on each platform. `content_parse` also reports `isControl`, so the harness can check §6.10's rule across engines instead of each engine checking itself. The matching harness stage is flamenet-e2e!27. Proven by breaking it: making this engine read the SPEC's `ment` instead of the shipped `men` turned "iPhone mentions parse on Android" red — the exact drift that has been invisible since the key was chosen. 1190 tests still pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 4.8
|
32e4af06aa |
Interop driver: stand up a peer on an older protocol version
`session_device` takes a `proto` so the harness can build a peer that advertises the previous version, which is the only way to keep the standing compatibility rule honest — the current version and the one before it must interoperate, and until now the suite could only test the newest code against itself. ⚠️ The capability signature is remade over the number actually claimed. A bundle that advertises one version and signs another is rejected by `provenVersion` and falls back to v1, so leaving the original signature would let a compatibility test pass while exercising a pairing nobody ships. The override lives in the driver rather than in `registrationWire`: the engine should publish what it implements, not what a test wants. `session_reload` gained `as_version` for the same reason — reproducing a session stored by an older build, which is a state no amount of running the new code can otherwise produce. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 4.8
|
2402f7a665 |
Interop driver: report the session version a responder settled on
`session_respond` returned no version at all, so the cross-engine test could not assert the one thing that mattered — that a session this device ANSWERS agrees with the initiator about which protocol it runs. `session_initiate` and `session_encrypt` report it too, so a version that changes across a store-and-reopen is visible rather than inferred. Sealed groups: file the message instead of dropping it in silence. `groupRoster` asked the relay for the membership of a group the relay cannot know — §8.9b exists so it holds opaque tags and no member list — so the sender was never "in the roster" and EVERY message in EVERY sealed group was discarded on arrival by the check meant to protect it. The sealed roster is the authoritative list and the only one that exists. A refusal now says which group and which sender. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Paul Hitt
|
9787e2e6c5 |
Correct a driver comment that outlived what it described
newDevice's header said this engine has no ML-KEM and therefore always publishes proto 1. It gained ML-KEM, the command gained a `pq` flag, and the comment stayed — so it now reads as a statement of a limitation that no longer exists, on the one file whose job is to be the honest account of what the Kotlin engine can do. Also names the distinction a reader of this file needs: these low-level commands wire X3DH and the ratchet together by hand, and the session_* commands go through E2ESession, which is the layer the app calls and the layer that owns the negotiation decision. |
||
|
Paul HittandClaude Opus 4.8
|
1e96af0122 |
A v2 peer was being sent a post-quantum ratchet step
Proving the v3 ratchet across engines turned up a bug in shipping it, and this one would have broken every existing Android conversation. `E2ESession.initiate` turned the v3 ratchet on whenever the peer offered a PQ prekey. Every v2 device offers one — offering a prekey is a claim to §12, not to §13. So a build with this change would attach `pqdh`/`pqct` to messages for 0.6, the released build, whose ratchet has no post-quantum anything. That fails twice over, which is why it would have looked like a network problem. Their root step derives without `ss` and under the `_v1` info string while ours used `_v3`, so the keys diverge; and their canonical header omits the two fields, so the AAD does not match either. The first message never opens and never would. Gated on a PROVEN claim now, the way the iOS engine already does it: `chosen != null && bundle.proto >= 3`. `bundle.proto` on its own is the relay's word for it, but `chosen` is non-null only when the capability signature over that same proto verified (§12.7), so the pair is a claim the peer actually made. The test asserts on the absence of a header field, which is the only way to catch this — every message still decrypts in a self-test, because both sides of one engine agree with each other about the mistake. The interop driver now publishes a real v3 bundle: `pqspk` under `FlamenetE2E_PQSPK_v2`, and `caps_sig` over `FlamenetE2E_Caps_v2 ‖ uint8(3)`. Both were needed before the web engine would negotiate anything but v1 — which is why the harness had been quietly proving v1 and nothing else. It can still publish a v1 bundle on request, because the fleet still holds devices that speak it and that coverage must not disappear the day v3 lands. 605 tests, 0 failures. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
a213ad8d5d |
Android: sealed sender, so the relay stops learning who talks to whom
Key transparency stopped the relay substituting keys. It did nothing about the relay watching who writes to whom, which is the metadata most people actually leak. §14 moves the sender's identity inside the ciphertext: the relay learns who a message is for and nothing about who it is from. Verified against relay1 rather than asserted. A sealed submission is accepted unauthenticated, the stored row comes back with `from_user = 0` — the relay saying it does not know — and the recipient still reads the message with the sender recovered from inside the seal. Cross-engine verified in BOTH directions against the normative web engine, and that mattered: the seal is asymmetric in practice, since one side derives from an ephemeral it generated and the other from one it was handed, so a byte-order or AAD mistake can easily break only one direction and a single-direction test picks the half that works. The interop harness gained `seal` and `open_sealed` ops for it; 43 assertions, none failing. 🔴 **The seal is confidentiality, never authentication.** The recipient's `IK_dh` is published to the world, so anyone can produce a well-formed seal claiming to be from anybody. The claimed sender picks a session and nothing else; the ratchet opening it is what makes it a fact, and the identity pin is what refuses an impostor. There is a test that forges a sender deliberately, so that if somebody later renames these fields into something confident the reason they are claims is still written down and still executing. ⚠️ **The first message to a new contact cannot be sealed.** Sealing needs the recipient's delivery key and that key travels inside an E2E message, so a prior exchange is required. `Sent.sealed` reports which happened rather than leaving the UI free to imply anonymity that was not achieved — a silent fall back to an unsealed send is exactly what §14.5 warns about. ⚠️ **Rotation is blunt, and the copy says so.** There is no sender to block, so the only lever is replacing the delivery key, which cuts off every contact until the next time they are written to. Reachable from Account, asserted by `CrudSurfaceTest`, and live-tested: a submission presenting a retired key is refused by the relay. The relay stores only `SHA-256(key)`. The key itself never passes through the relay from its owner, which is the only reason an unauthenticated sealed submission can be authorised without naming who is submitting. Tested: 108 engine, 67 client, 43 interop, none failing, live suites enabled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
d30a77dd1e |
Android: contacts, conversations, and a message that made the round trip
The app now holds a conversation. A message typed into it reaches a different
program through the real relay, that program answers, and the answer appears in
the thread -- verified three times, end to end, encrypted the whole way.
96 JVM tests, 37 cross-engine, 11 on-device, plus live round trips against the
throwaway fleet.
What landed: a fan-out send that reaches every device a person has, inbox
polling, message history encrypted at rest, contacts, presence, and a session
that survives a restart. Plus `:interop:peer` -- a Flamenet account you can
drive from a shell, so the correspondent on the other end is not this app
talking to itself.
Three wire shapes were wrong, and only a real relay could say so:
- `prekeys`, not `opks`. The relay answered 200 and stored nothing, so the
device was published and could not be started with.
- The send body names the recipient once at the top, not per envelope.
- The buddy list is `id`/`presence`/`screen_name`, not `user_id`/`status`.
The second and third at least failed loudly. The first did not, which is why
the live tests exist.
Decisions worth stating:
A send returns partial success rather than throwing. One of someone's three
devices being unreachable must not stop the message reaching the other two, and
the UI says "delivered to 2 of 3" rather than "sent".
The inbox watermark advances per message, after it is decrypted AND stored --
never once for the batch, which would lose everything after a crash. An
undecryptable message advances it too: it is never going to decrypt, and
leaving it would re-fail on every poll for ever.
A sent line is recorded only once a device took it. Writing it first produces a
thread showing messages the recipient never got, which is worse than an error --
the sender believes they said something.
Opening a thread marks it read locally and tells the relay nothing. A read
receipt is a disclosure, and this product does not make one by default.
History is keyed by person, not device. Someone's phone and tablet are one
conversation, and filing by device would split it into threads each holding
half the story.
Also improves how instrumented failures report: the assertion now prints what
the screen was showing. That turned a bare 90-second timeout into "No value for
user_id" and then "Failed to connect to relay1" -- one a bug, one the emulator's
network, and previously indistinguishable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Paul HittandClaude Opus 5
|
5f983c1455 |
Android: the app, and the two things only a phone could tell us
There is now a Flamenet Messenger app that installs, launches, creates an account on a self-hosted relay, registers a device, publishes prekeys, and stores all of it in an Android Keystore-backed vault. Verified on an emulator through the actual screens, not by unit test. 92 JVM tests, 37 cross-engine, 9 on-device, 3 against live relays. Two findings, both of which the JVM suite was structurally incapable of producing. **Android has no ordinary Ed25519 provider.** `KeyPairGenerator.Ed25519` and `KeyFactory.Ed25519` come from `AndroidKeyStore` and from nowhere else -- probed on API 35 rather than assumed -- and Keystore only makes keys it keeps. Every Ed25519 key in this engine is one we must hold: a device identity in our own vault, an account key derived from a passphrase. So the JCA route could never have worked. It compiled, passed 81 tests on a laptop, and threw "Not initialized" on the first launch on a phone. Worse, it was LATENT: a wrong password failed before any key was minted, so the error path worked perfectly while the happy path was broken. Ed25519 now uses BouncyCastle's `math.ec.rfc8032` core directly -- the engine's one real dependency, present because the platform leaves a hole, and a strictly better answer than the non-constant-time keygen it replaces. ProviderProbeTest records the measurement so nobody guesses again. **The prekey field is `prekeys`, not `opks`.** I guessed. The relay accepted the registration, returned 200, and stored nothing -- so the device was published and could not be started with, a failure with no error anywhere. Caught by a live test against the throwaway fleet, which exists because the same path through the emulator takes four minutes to say "timed out" and nothing about where. Two testing traps, recorded where they bite: An indeterminate progress spinner animates for ever, so Compose's test clock never goes idle and `waitUntil` never evaluates its condition -- it fails on the full timeout, reading as "sign-up is broken" when sign-up works. Auto-advance is now disabled around the wait only; disabling it for the whole test is the next mistake, because the app opens on a spinner while it reads the vault. Wiping the Keystore in `@Before` deletes the key the already-launched activity is holding, and every later write fails against a key that no longer exists. The test signs out through the UI instead, which is both safe and what a person does. Failure messages now include what the screen was showing. A bare timeout sends you to the network, the relay and the crypto in turn while the app has been displaying the reason the whole time. Also corrects a note that said this Mac had no Android SDK. It does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
8a3770f1d1 |
Kotlin: the session layer, verified across the wire
An app does not use X3DH and a ratchet, it uses "send this string to that person". This is that layer: device identities with prekeys and rotation, envelopes as the relay actually carries them, and sessions that survive being put away and taken out again. 11 unit tests, and the interop harness grows from 23 assertions to 37 -- the new ones at the envelope level, which is where this protocol's one shipped bug lived. A client and a relay disagreed about a field name and nothing caught it until a live round trip; hand-assembled arguments to X3DH would not have. Kotlin now builds a whole envelope, the web engine's own session layer parses it, and both directions agree on the associated data. Three decisions worth stating. Retired signed prekeys are kept for longer than the relay keeps undelivered mail. A sender fetches a bundle, encrypts against that prekey id, and the envelope can then sit in a queue for the relay's full retention window. Discarding the private half at rotation turns every one of those into an AEAD failure -- which the receiving client cannot distinguish from an attack. A prekey retired past retention is refused outright rather than guessed at, for the same reason: guessing derives a garbage secret and reports it as tampering. The handshake block rides every message until one is answered, not until one is sent. Any message may be the first to arrive. Sessions archive with their skipped message keys, and the harness proves it against the peer rather than against our own bytes. Those keys are the messages already in flight; an archive that drops them loses exactly the mail that was in the air when the app was killed. The failure this whole path exists to prevent is a client that loses session state, re-registers a device, and shows every contact "security keys changed" -- the warning that is supposed to mean an attack. `org.json` is compileOnly: it is in the Android framework, so it costs nothing on a phone and bundling a copy would shadow the platform's. Safe only because the two encodings that must match byte-for-byte -- the ratchet header and the canonical log leaf -- are built by hand; Android's implementation preserves key order and the reference one does not. Also adds PARITY.md: what the iOS app does, what Android does, what is left. It is the definition of done, and a row moves only when something tests it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
fc93df24e6 |
Kotlin: Ed25519, and cross-engine interop against the web engine
The seventeen Kotlin tests that landed with X3DH and the ratchet prove the engine agrees with itself. That is not the property anyone needs. Two engines written by the same person agree about the same misreadings of the spec, and every one of those shows up months later as "the message will not decrypt" on someone else's phone. So this wires the Kotlin engine up against the web engine, which is normative, in both directions. 23 assertions, all passing: v1 X3DH both ways, identical associated data, alternating ratchet turns past the handshake, out-of-order and permanently lost messages, a drained one-time prekey pool, tamper rejection, safety numbers, and the canonical transparency-log leaf. Three things worth recording. Ed25519 was missing, and the engine could derive an account address but never sign a claim for one. The JCA offers no route from a private EdDSA key to its public half, so the RFC 8032 keygen is done here -- and verified against the RFC 8032 test vectors, not against a round-trip through our own code. A round-trip proves only that the file agrees with itself, and every wrong clamp, endianness and sign convention agrees with itself perfectly. The scalar multiplication uses BigInteger and is therefore not constant-time; it runs once at sign-in, in the user's own process, and signing deliberately stays with the platform provider. The harness reports a build failure as a failure, never a skip. The Swift harness beside it once printed "skipped" -- which reads as green -- while silently building a different checkout. Only a missing JDK skips here, and the resolved path is printed either way. Each section is wrapped so a divergence is a named failure rather than a crash. That was not theoretical: breaking the Kotlin message-key derivation on purpose crashed the process with a stack trace and skipped every later section. It now reports four failing sections and finishes the run. The negative control is how this was checked at all -- a suite that passes on its first run and has never been seen to fail is not evidence of anything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |