d30a77dd1e432b9da3f820365ff807cdcd850195
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul HittandClaude Opus 5
|
d30a77dd1e |
Android: contacts, conversations, and a message that made the round trip
The app now holds a conversation. A message typed into it reaches a different
program through the real relay, that program answers, and the answer appears in
the thread -- verified three times, end to end, encrypted the whole way.
96 JVM tests, 37 cross-engine, 11 on-device, plus live round trips against the
throwaway fleet.
What landed: a fan-out send that reaches every device a person has, inbox
polling, message history encrypted at rest, contacts, presence, and a session
that survives a restart. Plus `:interop:peer` -- a Flamenet account you can
drive from a shell, so the correspondent on the other end is not this app
talking to itself.
Three wire shapes were wrong, and only a real relay could say so:
- `prekeys`, not `opks`. The relay answered 200 and stored nothing, so the
device was published and could not be started with.
- The send body names the recipient once at the top, not per envelope.
- The buddy list is `id`/`presence`/`screen_name`, not `user_id`/`status`.
The second and third at least failed loudly. The first did not, which is why
the live tests exist.
Decisions worth stating:
A send returns partial success rather than throwing. One of someone's three
devices being unreachable must not stop the message reaching the other two, and
the UI says "delivered to 2 of 3" rather than "sent".
The inbox watermark advances per message, after it is decrypted AND stored --
never once for the batch, which would lose everything after a crash. An
undecryptable message advances it too: it is never going to decrypt, and
leaving it would re-fail on every poll for ever.
A sent line is recorded only once a device took it. Writing it first produces a
thread showing messages the recipient never got, which is worse than an error --
the sender believes they said something.
Opening a thread marks it read locally and tells the relay nothing. A read
receipt is a disclosure, and this product does not make one by default.
History is keyed by person, not device. Someone's phone and tablet are one
conversation, and filing by device would split it into threads each holding
half the story.
Also improves how instrumented failures report: the assertion now prints what
the screen was showing. That turned a bare 90-second timeout into "No value for
user_id" and then "Failed to connect to relay1" -- one a bug, one the emulator's
network, and previously indistinguishable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Paul HittandClaude Opus 5
|
5f983c1455 |
Android: the app, and the two things only a phone could tell us
There is now a Flamenet Messenger app that installs, launches, creates an account on a self-hosted relay, registers a device, publishes prekeys, and stores all of it in an Android Keystore-backed vault. Verified on an emulator through the actual screens, not by unit test. 92 JVM tests, 37 cross-engine, 9 on-device, 3 against live relays. Two findings, both of which the JVM suite was structurally incapable of producing. **Android has no ordinary Ed25519 provider.** `KeyPairGenerator.Ed25519` and `KeyFactory.Ed25519` come from `AndroidKeyStore` and from nowhere else -- probed on API 35 rather than assumed -- and Keystore only makes keys it keeps. Every Ed25519 key in this engine is one we must hold: a device identity in our own vault, an account key derived from a passphrase. So the JCA route could never have worked. It compiled, passed 81 tests on a laptop, and threw "Not initialized" on the first launch on a phone. Worse, it was LATENT: a wrong password failed before any key was minted, so the error path worked perfectly while the happy path was broken. Ed25519 now uses BouncyCastle's `math.ec.rfc8032` core directly -- the engine's one real dependency, present because the platform leaves a hole, and a strictly better answer than the non-constant-time keygen it replaces. ProviderProbeTest records the measurement so nobody guesses again. **The prekey field is `prekeys`, not `opks`.** I guessed. The relay accepted the registration, returned 200, and stored nothing -- so the device was published and could not be started with, a failure with no error anywhere. Caught by a live test against the throwaway fleet, which exists because the same path through the emulator takes four minutes to say "timed out" and nothing about where. Two testing traps, recorded where they bite: An indeterminate progress spinner animates for ever, so Compose's test clock never goes idle and `waitUntil` never evaluates its condition -- it fails on the full timeout, reading as "sign-up is broken" when sign-up works. Auto-advance is now disabled around the wait only; disabling it for the whole test is the next mistake, because the app opens on a spinner while it reads the vault. Wiping the Keystore in `@Before` deletes the key the already-launched activity is holding, and every later write fails against a key that no longer exists. The test signs out through the UI instead, which is both safe and what a person does. Failure messages now include what the screen was showing. A bare timeout sends you to the network, the relay and the crypto in turn while the app has been displaying the reason the whole time. Also corrects a note that said this Mac had no Android SDK. It does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
b1ba9cdc58 |
Kotlin: the relay transport
The engine can hold a conversation but had no way to reach a relay. This is that: the §8 routes, both authentication modes, and the transparency log. 13 tests against a real HTTP server rather than a hand-written fake. That distinction is the point -- a fake written alongside the client agrees with it about everything including the field name it got wrong, which is exactly how this protocol's one shipped bug survived until a live round trip. Two more tests run against the deployed relay, opt-in via FN_LIVE_RELAY and silent when unset. A test that reaches the network cannot sit in a hermetic suite; it fails on an aeroplane and a suite that is red for unrelated reasons stops being read. What they check is the thing MockWebServer structurally cannot: that the URL this client builds is one relay.flamenet.io actually serves. It does, on both mounts, and the Kotlin engine verified an inclusion proof from the production log against a head fetched separately -- not against the root the proof response carried, because a server that supplies both halves of a comparison has proven nothing. Details that are load-bearing: The per-request signature covers the LOGICAL section-8 path, not the URL's. The relay mounts these routes twice and signs against the former, so signing the REST-prefixed path yields a valid signature over the wrong string and a refusal with no hint as to why. Tested by asserting the other one does NOT verify. The nonce is not decoration. Two inbox polls in the same second with the same empty body would otherwise produce identical signatures, and the relay's replay cache cannot tell an honest repeat from a captured one -- so it refuses the honest client and the app quietly stops receiving mail. Token minting is lazy, and a deployment with no token route is asked exactly once. Laziness cost an hour here: the tests hung, because takeRequest() blocks forever and I was waiting for a mint that had not happened yet. They now use a bounded read, so that mistake fails instead of looking like a hung build. Application passwords are stripped of spaces. The account service prints them in readable groups and users paste what they were shown; sending the spaces is a 401 nobody can diagnose, because the password on screen looks right. Kotlin nests block comments, so `/e2e/` followed by a star inside a doc comment opens one that never closes and the compiler blames the last line of the file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
8a3770f1d1 |
Kotlin: the session layer, verified across the wire
An app does not use X3DH and a ratchet, it uses "send this string to that person". This is that layer: device identities with prekeys and rotation, envelopes as the relay actually carries them, and sessions that survive being put away and taken out again. 11 unit tests, and the interop harness grows from 23 assertions to 37 -- the new ones at the envelope level, which is where this protocol's one shipped bug lived. A client and a relay disagreed about a field name and nothing caught it until a live round trip; hand-assembled arguments to X3DH would not have. Kotlin now builds a whole envelope, the web engine's own session layer parses it, and both directions agree on the associated data. Three decisions worth stating. Retired signed prekeys are kept for longer than the relay keeps undelivered mail. A sender fetches a bundle, encrypts against that prekey id, and the envelope can then sit in a queue for the relay's full retention window. Discarding the private half at rotation turns every one of those into an AEAD failure -- which the receiving client cannot distinguish from an attack. A prekey retired past retention is refused outright rather than guessed at, for the same reason: guessing derives a garbage secret and reports it as tampering. The handshake block rides every message until one is answered, not until one is sent. Any message may be the first to arrive. Sessions archive with their skipped message keys, and the harness proves it against the peer rather than against our own bytes. Those keys are the messages already in flight; an archive that drops them loses exactly the mail that was in the air when the app was killed. The failure this whole path exists to prevent is a client that loses session state, re-registers a device, and shows every contact "security keys changed" -- the warning that is supposed to mean an attack. `org.json` is compileOnly: it is in the Android framework, so it costs nothing on a phone and bundling a copy would shadow the platform's. Safe only because the two encodings that must match byte-for-byte -- the ratchet header and the canonical log leaf -- are built by hand; Android's implementation preserves key order and the reference one does not. Also adds PARITY.md: what the iOS app does, what Android does, what is left. It is the definition of done, and a row moves only when something tests it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
fc93df24e6 |
Kotlin: Ed25519, and cross-engine interop against the web engine
The seventeen Kotlin tests that landed with X3DH and the ratchet prove the engine agrees with itself. That is not the property anyone needs. Two engines written by the same person agree about the same misreadings of the spec, and every one of those shows up months later as "the message will not decrypt" on someone else's phone. So this wires the Kotlin engine up against the web engine, which is normative, in both directions. 23 assertions, all passing: v1 X3DH both ways, identical associated data, alternating ratchet turns past the handshake, out-of-order and permanently lost messages, a drained one-time prekey pool, tamper rejection, safety numbers, and the canonical transparency-log leaf. Three things worth recording. Ed25519 was missing, and the engine could derive an account address but never sign a claim for one. The JCA offers no route from a private EdDSA key to its public half, so the RFC 8032 keygen is done here -- and verified against the RFC 8032 test vectors, not against a round-trip through our own code. A round-trip proves only that the file agrees with itself, and every wrong clamp, endianness and sign convention agrees with itself perfectly. The scalar multiplication uses BigInteger and is therefore not constant-time; it runs once at sign-in, in the user's own process, and signing deliberately stays with the platform provider. The harness reports a build failure as a failure, never a skip. The Swift harness beside it once printed "skipped" -- which reads as green -- while silently building a different checkout. Only a missing JDK skips here, and the resolved path is printed either way. Each section is wrapped so a divergence is a named failure rather than a crash. That was not theoretical: breaking the Kotlin message-key derivation on purpose crashed the process with a stack trace and skipped every later section. It now reports four failing sections and finishes the run. The negative control is how this was checked at all -- a suite that passes on its first run and has never been seen to fail is not evidence of anything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
e0589eff14 |
Kotlin: the Double Ratchet
The second half of a protocol Android did not have. With X3DH it can now agree a secret, and with this it can hold a conversation. Two ratchets: the symmetric one advances per message so a compromised key reveals nothing earlier, the DH one advances on each direction change so a conversation recovers from compromise rather than staying broken. The tests run conversations rather than checking steps, because that is what catches the bugs this code actually has. X3DH is a pure function — it agrees or it does not. A ratchet can agree for six messages and diverge on the seventh because one side counted a skipped message differently, and the symptom is only that something will not decrypt. So: out-of-order delivery, a message lost forever, a gap ACROSS a direction change (which is the whole reason "pn" is on the wire), and four DH turns in sequence. Three things that would each produce plausible-but-wrong bytes, so each is tested rather than reasoned about: - the sending chain advances BEFORE the key is used, so a caller that retries cannot encrypt twice under one key — two GCM encryptions under one key and nonce leak the XOR of both plaintexts - the header's canonical JSON is hand-built, key order "dh, pn, n", no spaces, because it goes into the AAD and three implementations must reproduce it byte-for-byte - the derived nonce is stripped from the wire rather than sent, so a receiver cannot be handed a nonce that disagrees with the one its key implies MAX_SKIP caps derivation at 1000: a header claiming a huge counter costs the sender nothing and the receiver everything. 13 tests. Still missing before an app: a session layer, a wire client, and the app itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
c893d412d2 |
Kotlin: X3DH, and the account KDF beneath it
The Android engine had primitives and no protocol: no handshake, no ratchet, no session. It could verify a safety number and could not start a conversation. X3DH is the first half of fixing that. The four agreements are mixed in the order the spec fixes, and DH4 is omitted rather than zero-filled when the one-time prekey pool is drained. Both are choices that produce 32 plausible bytes when wrong — the failure lands at the far end as a message that will not decrypt, pointing nowhere near here, so the tests check that the two sides AGREE rather than that either looks right. Keys are JCA objects rather than raw bytes throughout, because Conscrypt will not build an XDH key from raw bytes and wants a DER spec. Passing raw bytes would work on this JVM and throw on the first tap on a phone, which this engine has already been bitten by once. Also here: AccountKdf, so Android can derive an account key and an address. Checked against the vectors pinned from the enclave client — including the non-ASCII and empty-passphrase cases, where a CharArray API and a UTF-8 wire format quietly disagree. ⚠️ Still missing before an Android app can send a message: the Double Ratchet, a session layer, a wire client, and the app itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul Hitt
|
b7021e9a73 |
Merge remote-tracking branch 'origin/main' into feat/android-account-kdf
# Conflicts: # Sources/Views/SignInView.swift |
||
|
Paul HittandClaude Opus 5
|
6f83248e1c |
Give an account an identity no server issued
A relay's `user_id` is a row number it assigned itself, and the account service's pseudonym is HMAC'd per product precisely so two services cannot correlate their users. Neither survives a trip to a second relay, so nothing addressed by either can cross a relay boundary — which is the first thing federation needs, and the reason a deleted relay currently takes its users' names with it. So identity becomes a key the client derives and holds: Ed25519, stretched from the passphrase under `flamenet/account/identity/v1`, alongside the per-product keys that already work this way. The public half, Crockford base32, is the address. Derived rather than generated, so the same passphrase reproduces the same fnid on a device that never held it — re-register the same key at a new relay and contacts who pinned it are still talking to the same person. Relay: `users.account_key` / `users.fnid` (unique in the database, not just in the handler), `POST /e2e/account/identity` to claim by proving possession, and `GET /e2e/account/identity/:handle` to resolve. The claim signature binds the audience, the local user id and the key itself; dropping any one of the three lets a claim be replayed at another relay, lifted onto another account here, or transplanted onto another key. Verification uses the very key being registered, so the relay confirms possession and never acquires an opinion about whose key it is. Rotation is refused rather than silently overwritten — a new address strands every contact that pinned the old one, and that needs a migration record they can follow. Log leaves go v1 -> v2, binding a subject instead of a row number: `fnid:<b32>` where an account has enrolled, `local:<audience>/<user_id>` where it has not. The fallback is qualified rather than bare because a device can be registered before its key is claimed, and the alternatives were worse in both directions — skipping those devices leaves a hole in the one record that must not have holes, and logging them unqualified produces leaves that mean different things depending on origin with nothing in the leaf to say so. This is not a server-side change. All three engines hash the leaf and two of them verify it against the relay, and a published vector pins the exact bytes, so the format moves in one commit or not at all. Existing v1 leaves keep their hashes and are backfilled to a self-describing `legacy-v1/` subject: the log is append-only, and a format change is not a licence to rewrite history. The version prefix sits inside the hashed bytes so a v1 leaf recomputed under v2 rules fails rather than verifying against a different subject. Green: engine conformance 80, relay §8 suite 235 (10 new), JS<->Swift interop 74, Kotlin published vectors. Two things found on the way, neither addressed here: - iOS never calls `enforceKeyLog`. It and `requireKeyLog` are both declared in E2EInbox and nothing reads either, so §19 is dead code on that client while the JS one wires it. The subject is threaded through ready for a caller. - `engine/test/interop/build.sh` prefers ~/Code/flamenetmessenger over the checkout it is run from, so in a worktree it silently builds the wrong sources and reports a *skip* rather than a failure. Set FNMSG_APP_DIR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
85f6a29953 |
Kotlin: X25519, with the small-order check that is not optional here
Raw 32-byte keys on the wire, X.509 SubjectPublicKeyInfo in the JCA -- so every crossing needs the 12-byte DER prefix added or stripped. Written out as a constant rather than parsed: a DER parser here would be more code than the thing it parses, and one more place to get a length wrong. Refuses an all-zero shared secret. X25519 produces all zeros for a small-order peer key, and a peer that sends one forces both sides to a secret it already knows. RFC 7748 says implementations MAY check; when the peer's key arrives over a relay nobody trusts, MAY is not the right reading. All four small-order points from §6.1 are tested. ⚠️ XDH is API 33+ on Android. Older devices throw NoSuchAlgorithmException at the first call, and the app must either bundle a provider or refuse to run -- a decision nobody has made, so it is documented rather than papered over. ⚠️ These run on the JVM against SunEC. Android routes the same calls through Conscrypt. Passing here says the encoding is right and nothing about a device. Kotlin engine: 27 tests -> 33. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
bff7a4bbdd |
Kotlin: HKDF-SHA256 and AES-256-GCM, against external vectors
Checked against RFC 5869 and NIST CAVP rather than against our JavaScript engine. An implementation validated only against our other implementation inherits our misunderstandings, and finding where the first two quietly agreed on something the spec does not say is the entire reason for writing a third. Three things pinned that a naive version gets wrong: - An empty HKDF salt means HashLen zeros, not "no HMAC key". SecretKeySpec throws on a zero-LENGTH key, so the naive version crashes on exactly the case the app uses -- §12 derives its per-context secrets with an empty salt. - 255 blocks is a hard ceiling. The counter is one byte; a naive loop wraps at 256 and silently returns key material it already returned. - The AEAD box is nonce || ciphertext || tag. CryptoKit's `combined` emits that layout and the JS engine assembles it by hand to match, so a box built any other way decrypts to garbage on the other engines rather than failing loudly. HKDF is hand-written on Mac rather than taken from a provider: Android's providers disagree about whether HKDF exists at all. Kotlin engine: 9 tests -> 27. ⚠️ Still JVM-only. These prove nothing about Android, where crypto goes through Conscrypt rather than the JVM's JCA, and nothing here has ever been compiled against the Android SDK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
e7862c0d95 |
Point the site and the harnesses at the four repositories
The relay and the Kotlin engine are published on their own now, so the self-hosting guide told people to clone a directory that no longer exists and the source page said everything was in one repository. Both pages list all four now, with what each is for. The self-hosting guide still only needs the first one -- the rest are there so that self-hostable is something a reader can check rather than something they are told. The Android vector loader hardcoded ../../engine/test/vectors, which is only right inside this repository. It resolves from a candidate list now, honours FLAMENET_VECTORS, and still FAILS rather than skips when it finds nothing: the entire value of a third engine is that it is checked against something other than a reading of our own code. Nine tests pass against the JDK in Homebrew, which is not on the default java_home path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
06b97ca69b |
Start Android with the engine, held to the published vectors
Android does not exist, and the first thing it needs is not a screen. It needs the protocol implemented a third time — which is only worth anything if the third implementation is checked against the spec and the published vectors rather than against a reading of our own Swift. So `android/engine` is a plain Kotlin/JVM library with no Android in it: no SDK, no emulator, no device, and therefore gateable in CI on Linux like everything else. It reads engine/test/vectors/protocol-v1.json from its one location rather than carrying a copy, because a vector transcribed into the test that checks it is a vector that agrees with itself. Covered so far: safety numbers (§7) and key transparency verification (§19) — RFC 6962 leaf and node hashing, the unbalanced split, and bottom-up audit paths. Nine tests, all green against the same file the JavaScript and Swift engines answer to. The §19 tests are the ones worth having. A top-down reading of a bottom-up audit path yields the correct root for every left-spine index, so it passes any suite that only checks index 0 — the vectors pin a right-descending index, and the tests assert both that the published path verifies and that the same path reversed does not. android/README.md lists what is missing — X25519, X3DH, the ratchet, sealed sender, both post-quantum halves, the relay client, the app — as a table rather than as an omission. Android stays honestly marked "not started" on the site until that list is shorter. Bytecode targets 17 without demanding a JDK 17 be installed: a jvmToolchain(17) fails on any newer JDK, which is a poor trade for a library with no JDK 17-specific behaviour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |