8a3770f1d14a48eb64500d2b52392cdd3d7a2d61
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul HittandClaude Opus 5
|
8a3770f1d1 |
Kotlin: the session layer, verified across the wire
An app does not use X3DH and a ratchet, it uses "send this string to that person". This is that layer: device identities with prekeys and rotation, envelopes as the relay actually carries them, and sessions that survive being put away and taken out again. 11 unit tests, and the interop harness grows from 23 assertions to 37 -- the new ones at the envelope level, which is where this protocol's one shipped bug lived. A client and a relay disagreed about a field name and nothing caught it until a live round trip; hand-assembled arguments to X3DH would not have. Kotlin now builds a whole envelope, the web engine's own session layer parses it, and both directions agree on the associated data. Three decisions worth stating. Retired signed prekeys are kept for longer than the relay keeps undelivered mail. A sender fetches a bundle, encrypts against that prekey id, and the envelope can then sit in a queue for the relay's full retention window. Discarding the private half at rotation turns every one of those into an AEAD failure -- which the receiving client cannot distinguish from an attack. A prekey retired past retention is refused outright rather than guessed at, for the same reason: guessing derives a garbage secret and reports it as tampering. The handshake block rides every message until one is answered, not until one is sent. Any message may be the first to arrive. Sessions archive with their skipped message keys, and the harness proves it against the peer rather than against our own bytes. Those keys are the messages already in flight; an archive that drops them loses exactly the mail that was in the air when the app was killed. The failure this whole path exists to prevent is a client that loses session state, re-registers a device, and shows every contact "security keys changed" -- the warning that is supposed to mean an attack. `org.json` is compileOnly: it is in the Android framework, so it costs nothing on a phone and bundling a copy would shadow the platform's. Safe only because the two encodings that must match byte-for-byte -- the ratchet header and the canonical log leaf -- are built by hand; Android's implementation preserves key order and the reference one does not. Also adds PARITY.md: what the iOS app does, what Android does, what is left. It is the definition of done, and a row moves only when something tests it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
fc93df24e6 |
Kotlin: Ed25519, and cross-engine interop against the web engine
The seventeen Kotlin tests that landed with X3DH and the ratchet prove the engine agrees with itself. That is not the property anyone needs. Two engines written by the same person agree about the same misreadings of the spec, and every one of those shows up months later as "the message will not decrypt" on someone else's phone. So this wires the Kotlin engine up against the web engine, which is normative, in both directions. 23 assertions, all passing: v1 X3DH both ways, identical associated data, alternating ratchet turns past the handshake, out-of-order and permanently lost messages, a drained one-time prekey pool, tamper rejection, safety numbers, and the canonical transparency-log leaf. Three things worth recording. Ed25519 was missing, and the engine could derive an account address but never sign a claim for one. The JCA offers no route from a private EdDSA key to its public half, so the RFC 8032 keygen is done here -- and verified against the RFC 8032 test vectors, not against a round-trip through our own code. A round-trip proves only that the file agrees with itself, and every wrong clamp, endianness and sign convention agrees with itself perfectly. The scalar multiplication uses BigInteger and is therefore not constant-time; it runs once at sign-in, in the user's own process, and signing deliberately stays with the platform provider. The harness reports a build failure as a failure, never a skip. The Swift harness beside it once printed "skipped" -- which reads as green -- while silently building a different checkout. Only a missing JDK skips here, and the resolved path is printed either way. Each section is wrapped so a divergence is a named failure rather than a crash. That was not theoretical: breaking the Kotlin message-key derivation on purpose crashed the process with a stack trace and skipped every later section. It now reports four failing sections and finishes the run. The negative control is how this was checked at all -- a suite that passes on its first run and has never been seen to fail is not evidence of anything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
e0589eff14 |
Kotlin: the Double Ratchet
The second half of a protocol Android did not have. With X3DH it can now agree a secret, and with this it can hold a conversation. Two ratchets: the symmetric one advances per message so a compromised key reveals nothing earlier, the DH one advances on each direction change so a conversation recovers from compromise rather than staying broken. The tests run conversations rather than checking steps, because that is what catches the bugs this code actually has. X3DH is a pure function — it agrees or it does not. A ratchet can agree for six messages and diverge on the seventh because one side counted a skipped message differently, and the symptom is only that something will not decrypt. So: out-of-order delivery, a message lost forever, a gap ACROSS a direction change (which is the whole reason "pn" is on the wire), and four DH turns in sequence. Three things that would each produce plausible-but-wrong bytes, so each is tested rather than reasoned about: - the sending chain advances BEFORE the key is used, so a caller that retries cannot encrypt twice under one key — two GCM encryptions under one key and nonce leak the XOR of both plaintexts - the header's canonical JSON is hand-built, key order "dh, pn, n", no spaces, because it goes into the AAD and three implementations must reproduce it byte-for-byte - the derived nonce is stripped from the wire rather than sent, so a receiver cannot be handed a nonce that disagrees with the one its key implies MAX_SKIP caps derivation at 1000: a header claiming a huge counter costs the sender nothing and the receiver everything. 13 tests. Still missing before an app: a session layer, a wire client, and the app itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
c893d412d2 |
Kotlin: X3DH, and the account KDF beneath it
The Android engine had primitives and no protocol: no handshake, no ratchet, no session. It could verify a safety number and could not start a conversation. X3DH is the first half of fixing that. The four agreements are mixed in the order the spec fixes, and DH4 is omitted rather than zero-filled when the one-time prekey pool is drained. Both are choices that produce 32 plausible bytes when wrong — the failure lands at the far end as a message that will not decrypt, pointing nowhere near here, so the tests check that the two sides AGREE rather than that either looks right. Keys are JCA objects rather than raw bytes throughout, because Conscrypt will not build an XDH key from raw bytes and wants a DER spec. Passing raw bytes would work on this JVM and throw on the first tap on a phone, which this engine has already been bitten by once. Also here: AccountKdf, so Android can derive an account key and an address. Checked against the vectors pinned from the enclave client — including the non-ASCII and empty-passphrase cases, where a CharArray API and a UTF-8 wire format quietly disagree. ⚠️ Still missing before an Android app can send a message: the Double Ratchet, a session layer, a wire client, and the app itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul Hitt
|
b7021e9a73 |
Merge remote-tracking branch 'origin/main' into feat/android-account-kdf
# Conflicts: # Sources/Views/SignInView.swift |
||
|
Paul HittandClaude Opus 5
|
6f83248e1c |
Give an account an identity no server issued
A relay's `user_id` is a row number it assigned itself, and the account service's pseudonym is HMAC'd per product precisely so two services cannot correlate their users. Neither survives a trip to a second relay, so nothing addressed by either can cross a relay boundary — which is the first thing federation needs, and the reason a deleted relay currently takes its users' names with it. So identity becomes a key the client derives and holds: Ed25519, stretched from the passphrase under `flamenet/account/identity/v1`, alongside the per-product keys that already work this way. The public half, Crockford base32, is the address. Derived rather than generated, so the same passphrase reproduces the same fnid on a device that never held it — re-register the same key at a new relay and contacts who pinned it are still talking to the same person. Relay: `users.account_key` / `users.fnid` (unique in the database, not just in the handler), `POST /e2e/account/identity` to claim by proving possession, and `GET /e2e/account/identity/:handle` to resolve. The claim signature binds the audience, the local user id and the key itself; dropping any one of the three lets a claim be replayed at another relay, lifted onto another account here, or transplanted onto another key. Verification uses the very key being registered, so the relay confirms possession and never acquires an opinion about whose key it is. Rotation is refused rather than silently overwritten — a new address strands every contact that pinned the old one, and that needs a migration record they can follow. Log leaves go v1 -> v2, binding a subject instead of a row number: `fnid:<b32>` where an account has enrolled, `local:<audience>/<user_id>` where it has not. The fallback is qualified rather than bare because a device can be registered before its key is claimed, and the alternatives were worse in both directions — skipping those devices leaves a hole in the one record that must not have holes, and logging them unqualified produces leaves that mean different things depending on origin with nothing in the leaf to say so. This is not a server-side change. All three engines hash the leaf and two of them verify it against the relay, and a published vector pins the exact bytes, so the format moves in one commit or not at all. Existing v1 leaves keep their hashes and are backfilled to a self-describing `legacy-v1/` subject: the log is append-only, and a format change is not a licence to rewrite history. The version prefix sits inside the hashed bytes so a v1 leaf recomputed under v2 rules fails rather than verifying against a different subject. Green: engine conformance 80, relay §8 suite 235 (10 new), JS<->Swift interop 74, Kotlin published vectors. Two things found on the way, neither addressed here: - iOS never calls `enforceKeyLog`. It and `requireKeyLog` are both declared in E2EInbox and nothing reads either, so §19 is dead code on that client while the JS one wires it. The subject is threaded through ready for a caller. - `engine/test/interop/build.sh` prefers ~/Code/flamenetmessenger over the checkout it is run from, so in a worktree it silently builds the wrong sources and reports a *skip* rather than a failure. Set FNMSG_APP_DIR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
85f6a29953 |
Kotlin: X25519, with the small-order check that is not optional here
Raw 32-byte keys on the wire, X.509 SubjectPublicKeyInfo in the JCA -- so every crossing needs the 12-byte DER prefix added or stripped. Written out as a constant rather than parsed: a DER parser here would be more code than the thing it parses, and one more place to get a length wrong. Refuses an all-zero shared secret. X25519 produces all zeros for a small-order peer key, and a peer that sends one forces both sides to a secret it already knows. RFC 7748 says implementations MAY check; when the peer's key arrives over a relay nobody trusts, MAY is not the right reading. All four small-order points from §6.1 are tested. ⚠️ XDH is API 33+ on Android. Older devices throw NoSuchAlgorithmException at the first call, and the app must either bundle a provider or refuse to run -- a decision nobody has made, so it is documented rather than papered over. ⚠️ These run on the JVM against SunEC. Android routes the same calls through Conscrypt. Passing here says the encoding is right and nothing about a device. Kotlin engine: 27 tests -> 33. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
bff7a4bbdd |
Kotlin: HKDF-SHA256 and AES-256-GCM, against external vectors
Checked against RFC 5869 and NIST CAVP rather than against our JavaScript engine. An implementation validated only against our other implementation inherits our misunderstandings, and finding where the first two quietly agreed on something the spec does not say is the entire reason for writing a third. Three things pinned that a naive version gets wrong: - An empty HKDF salt means HashLen zeros, not "no HMAC key". SecretKeySpec throws on a zero-LENGTH key, so the naive version crashes on exactly the case the app uses -- §12 derives its per-context secrets with an empty salt. - 255 blocks is a hard ceiling. The counter is one byte; a naive loop wraps at 256 and silently returns key material it already returned. - The AEAD box is nonce || ciphertext || tag. CryptoKit's `combined` emits that layout and the JS engine assembles it by hand to match, so a box built any other way decrypts to garbage on the other engines rather than failing loudly. HKDF is hand-written on Mac rather than taken from a provider: Android's providers disagree about whether HKDF exists at all. Kotlin engine: 9 tests -> 27. ⚠️ Still JVM-only. These prove nothing about Android, where crypto goes through Conscrypt rather than the JVM's JCA, and nothing here has ever been compiled against the Android SDK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
e7862c0d95 |
Point the site and the harnesses at the four repositories
The relay and the Kotlin engine are published on their own now, so the self-hosting guide told people to clone a directory that no longer exists and the source page said everything was in one repository. Both pages list all four now, with what each is for. The self-hosting guide still only needs the first one -- the rest are there so that self-hostable is something a reader can check rather than something they are told. The Android vector loader hardcoded ../../engine/test/vectors, which is only right inside this repository. It resolves from a candidate list now, honours FLAMENET_VECTORS, and still FAILS rather than skips when it finds nothing: the entire value of a third engine is that it is checked against something other than a reading of our own code. Nine tests pass against the JDK in Homebrew, which is not on the default java_home path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul HittandClaude Opus 5
|
06b97ca69b |
Start Android with the engine, held to the published vectors
Android does not exist, and the first thing it needs is not a screen. It needs the protocol implemented a third time — which is only worth anything if the third implementation is checked against the spec and the published vectors rather than against a reading of our own Swift. So `android/engine` is a plain Kotlin/JVM library with no Android in it: no SDK, no emulator, no device, and therefore gateable in CI on Linux like everything else. It reads engine/test/vectors/protocol-v1.json from its one location rather than carrying a copy, because a vector transcribed into the test that checks it is a vector that agrees with itself. Covered so far: safety numbers (§7) and key transparency verification (§19) — RFC 6962 leaf and node hashing, the unbalanced split, and bottom-up audit paths. Nine tests, all green against the same file the JavaScript and Swift engines answer to. The §19 tests are the ones worth having. A top-down reading of a bottom-up audit path yields the correct root for every left-spine index, so it passes any suite that only checks index 0 — the vectors pin a right-descending index, and the tests assert both that the published path verifies and that the same path reversed does not. android/README.md lists what is missing — X25519, X3DH, the ratchet, sealed sender, both post-quantum halves, the relay client, the app — as a table rather than as an omission. Android stays honestly marked "not started" on the site until that list is shorter. Bytecode targets 17 without demanding a JDK 17 be installed: a jvmToolchain(17) fails on any newer JDK, which is a poor trade for a library with no JDK 17-specific behaviour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |