diff --git a/PARITY.md b/PARITY.md new file mode 100644 index 0000000..4aa23ee --- /dev/null +++ b/PARITY.md @@ -0,0 +1,140 @@ +# Android ↔ iOS parity + +What the iOS app does, what Android does, and what is left. Read from the +sources listed, not from memory — every row names the iOS file it came from so +it can be re-checked rather than trusted. + +**Standing rule: this file is the definition of done for the Android app.** A +row moves to ✅ when the behaviour exists *and* something tests it, not when the +screen renders. + +--- + +## Where things stand + +| | iOS | Android | +|---|---|---| +| Protocol engine | `Sources/E2E`, 15 files | `engine`, 10 files | +| App layer | ~9,700 lines, 45 files | none yet | +| Tests | XCUITest + JS interop | 67 unit + 37 cross-engine interop | +| Ships to | nobody (no signing team) | nobody (no app module yet) | + +Android's engine is **verified against the web engine in both directions** +(`engine/test/interop-kotlin.mjs`), which is a stronger claim than the iOS app +could make for most of its history. What Android has no version of at all is +everything above the protocol. + +--- + +## Protocol engine + +| Feature | iOS | Android | Note | +|---|:--:|:--:|---| +| X25519, HKDF, AES-GCM | ✅ | ✅ | Android verified against RFC 5869 / NIST vectors | +| Ed25519 | ✅ | ✅ | RFC 8032 vectors; keygen-from-seed done here, signing via the provider | +| X3DH v1 | ✅ | ✅ | cross-engine verified | +| Double Ratchet | ✅ | ✅ | cross-engine verified, including reordering and loss | +| Safety numbers | ✅ | ✅ | cross-engine verified | +| Transparency log (`KeyLog`) | ✅ | ✅ | canonical leaf + inclusion proofs, cross-engine verified | +| Account KDF / fnid | ✅ | ✅ | both pinned to the published vector file | +| Sealed sender | ✅ | ⚪ | `E2E/SealedSender.swift` | +| Envelope + wire codec | ✅ | ✅ | cross-engine verified at the envelope level | +| Session archive / restore | ✅ | ✅ | cross-engine verified, in-flight messages included | +| Device identity + prekey rotation | ✅ | ✅ | retired prekeys retained past the relay window | +| Vault (encryption at rest) | ✅ | ⚪ | `E2E/KeychainE2EStore.swift` → Android Keystore | +| Attachment encryption | ✅ | ⚪ | `E2E/E2EAttachment.swift` | +| ML-KEM-768 (PQ agreement) | ⚠️ | ⚪ | see below | +| ML-DSA-65 (PQ identity) | ⚠️ | ⚪ | see below | + +⚠️ **The post-quantum gap is not what it looks like.** iOS has `PQKEM.swift` and +`PQSign.swift`, and they are inert: CryptoKit exposes ML-KEM only on iOS 26 and +the app targets iOS 16, so no shipped iOS build does post-quantum anything. +Android has no platform ML-KEM at any API level. So the two clients are +*currently equal in effect* and the honest parity target is v1/v2 negotiation +that degrades cleanly — which the Android engine already does, and the interop +harness asserts. Closing this properly is a separate decision (implement, or +take a vetted library) and it should be made once, for both clients. + +--- + +## Accounts and identity + +From `Sources/Auth/*` and `Sources/Views/Sign*`. + +| Feature | Android | +|---|:--:| +| Sign up (managed, shared self-hosted, local self-hosted) | ⚪ | +| Sign in, session restore | ⚪ | +| Sign out | ⚪ | +| Password reset request + confirm | ⚪ | +| Change password | ⚪ | +| Delete account | ⚪ | +| Relay picker / `SiteConfig` equivalent | ⚪ | +| Deterministic local-relay KDF salt | ⚪ | +| Multi-homing (publish + read home records) | ⚪ | + +⚠️ Sign-up is where the three deployment modes become visible to a user, and +each derives its salt differently. Getting this wrong does not error — it +derives a *different account* from the same passphrase. + +--- + +## Messaging + +From `E2E/E2EInbox.swift`, which is the real feature list. + +| Feature | Android | +|---|:--:| +| Send / receive text | ⚪ | +| Inbox polling | ⚪ | +| Reactions | ⚪ | +| Photos | ⚪ | +| Voice notes (record + play) | ⚪ | +| Stories, and seen-state | ⚪ | +| Groups: create, list, members, leave | ⚪ | +| Group text and images | ⚪ | +| Delivery keys + rotation | ⚪ | +| Trust state, verification, identity-change acceptance | ⚪ | +| Key-log enforcement and head gossip | ⚪ | +| Clear thread / all threads / caches | ⚪ | +| History at rest (encrypted, device-only key) | ⚪ | +| Presence and status | ⚪ | +| Buddy list, add, remove, block | ⚪ | +| Member lookup | ⚪ | +| Buddy commands | ⚪ | +| Voice calls (WebRTC, DTLS-SRTP, TURN) | ⚪ | +| Notifications list, read state | ⚪ | +| Services (AIM/ICQ via `flamenet-oscar`) | ⚪ | + +⚠️ Calls need a WebRTC dependency on Android as they do on iOS. That is the one +place parity forces a third-party library rather than a choice about one. + +--- + +## Settings + +Every key in `Views/Settings/SettingsKeys.swift` has to exist, because each one +is a promise the UI already makes: push on/off, per-kind notification toggles, +in-app sounds, previews, read receipts, last-seen, online status, app lock and +its grace period, send-on-return, timestamps, larger type, appearance, recent +emoji, photo auto-download. + +App lock also needs a lock screen that covers **everything including sheets**, +and obscures on backgrounding rather than only on return. + +--- + +## Build order + +Each step is useless without the one before it, which is why they are in this +order and not a nicer-looking one. + +1. ~~**Session layer** — envelope codec, device identity, archive/restore.~~ ✅ + Cross-engine verified at the envelope level, restarts included. +2. **Vault** — Android Keystore, encryption at rest for the archive above. +3. **`:client`** — REST, relay auth, device registration, prekey upload, inbox + polling, attachments, groups. +4. **`:app`** — Compose UI, in the order above: sign-in, buddy list, chat, then + the rest. +5. **Instrumented tests** — the JVM tests prove nothing about Conscrypt. This + engine has already been bitten once by exactly that. diff --git a/engine/build.gradle.kts b/engine/build.gradle.kts index 5bf283b..6e28df4 100644 --- a/engine/build.gradle.kts +++ b/engine/build.gradle.kts @@ -6,10 +6,17 @@ repositories { mavenCentral() } dependencies { testImplementation(kotlin("test")) - // Only to read the published vector file. The engine itself has no - // dependencies at all so far, which is worth keeping for as long as it - // lasts: every jar here is one more thing between the protocol and someone - // auditing it. + // `org.json` is part of the Android framework, so on a phone this costs + // zero shipped bytes and is `compileOnly` for that reason -- bundling a + // second copy would shadow the platform's and is a classic duplicate-class + // build failure. On the JVM (tests, the interop driver) it has to be a real + // dependency. + // + // Safe here only because nothing security-relevant depends on key order: + // Android's implementation preserves insertion order and the reference one + // does not, and the two bytes-must-match encodings in this engine -- the + // ratchet header and the canonical log leaf -- are both built by hand. + compileOnly("org.json:json:20240303") testImplementation("org.json:json:20240303") } diff --git a/engine/src/main/kotlin/io/flamenet/e2e/DeviceIdentity.kt b/engine/src/main/kotlin/io/flamenet/e2e/DeviceIdentity.kt new file mode 100644 index 0000000..db12720 --- /dev/null +++ b/engine/src/main/kotlin/io/flamenet/e2e/DeviceIdentity.kt @@ -0,0 +1,218 @@ +package io.flamenet.e2e + +import org.json.JSONObject +import java.security.KeyPair + +/** + * One device's long-term keys and its published prekeys (spec §3). + * + * A device has two identity keys, and conflating them is the classic mistake: + * [identityDH] does key agreement and [identitySigning] does signatures. + * X25519 and Ed25519 are related curves, but a single key used for both invites + * cross-protocol attacks and neither JCA nor CryptoKit will hand you the same + * object for the two jobs anyway. + * + * ⚠️ **Retired signed prekeys must be kept, not discarded.** A sender fetches a + * bundle, encrypts against that `spk_id`, and the envelope can then sit + * undelivered for the relay's full 30-day retention window. Deleting the + * private half at rotation turns every one of those queued messages into an + * AEAD failure — which is indistinguishable, to the receiving client, from an + * attack. That is why [retiredSignedPreKeys] exists and why the retention + * window here is longer than the relay's. + * + * A port of `engine/src/e2e.js`, which is normative. No post-quantum material + * yet: this engine has no ML-KEM, so it publishes a v1 bundle and the peer's + * `negotiate()` sees a v1 device, which is true. + */ +class DeviceIdentity private constructor( + val deviceId: String, + val identityDH: KeyPair, + val identitySigning: KeyPair, + var signedPreKey: KeyPair, + var signedPreKeyId: Int, + var signedPreKeyCreatedAt: Long, +) { + + /** A retired signed prekey, and when it stopped being published. */ + private class Retired(val keyPair: KeyPair, val retiredAt: Long) + + private val retiredSignedPreKeys = LinkedHashMap() + private val oneTimePreKeys = LinkedHashMap() + private var nextPreKeyId = 1 + + /** One published prekey: the id a sender quotes back, and the public half. */ + data class PreKey(val id: Int, val publicKey: ByteArray) + + companion object { + /** The spec's "every few weeks", made a number so something can enforce it. */ + const val ROTATE_AFTER_MS = 30L * 24 * 60 * 60 * 1000 + + /** + * How long a retired prekey stays usable. **Must exceed the relay's + * undelivered-envelope retention** (30 days by default), with margin + * for a sender that fetched a bundle and then went offline. Too short + * does not fail loudly — it drops old messages. + */ + const val RETAIN_AFTER_MS = 45L * 24 * 60 * 60 * 1000 + + fun create(now: Long = System.currentTimeMillis()): DeviceIdentity = DeviceIdentity( + deviceId = java.util.UUID.randomUUID().toString().lowercase(), + identityDH = X25519.generate(), + identitySigning = Ed25519.generate(), + signedPreKey = X25519.generate(), + signedPreKeyId = 1, + signedPreKeyCreatedAt = now, + ) + + fun fromArchive(archive: JSONObject): DeviceIdentity { + fun x(o: JSONObject) = KeyPair( + X25519.publicKeyFromRaw(Bytes.unb64(o.getString("pub"))), + X25519.privateKeyFromPkcs8(Bytes.unb64(o.getString("priv"))), + ) + val identity = DeviceIdentity( + deviceId = archive.getString("deviceId"), + identityDH = x(archive.getJSONObject("ikDH")), + identitySigning = Ed25519.fromSeed( + Bytes.unb64(archive.getJSONObject("ikSig").getString("seed"))), + signedPreKey = x(archive.getJSONObject("spk")), + signedPreKeyId = archive.getInt("spkId"), + // 0 means "written before this field existed", and the honest + // reading of an unknown age is "old" — a device that rotates + // once on first launch after an upgrade is harmless, one that + // never rotates because it defaulted to `now` is not. + signedPreKeyCreatedAt = archive.optLong("spkAt", 0), + ) + identity.nextPreKeyId = archive.optInt("nextId", 1) + + val otp = archive.optJSONObject("otp") ?: JSONObject() + for (key in otp.keys()) identity.oneTimePreKeys[key.toInt()] = x(otp.getJSONObject(key)) + + val previous = archive.optJSONObject("prevSpk") ?: JSONObject() + for (key in previous.keys()) { + val entry = previous.getJSONObject(key) + identity.retiredSignedPreKeys[key.toInt()] = + Retired(x(entry), entry.optLong("retiredAt", 0)) + } + return identity + } + } + + val identityDHPublic: ByteArray get() = X25519.rawFromPublicKey(identityDH.public) + val identitySigningPublic: ByteArray get() = Ed25519.rawFromPublicKey(identitySigning.public) + + /** + * The signature a peer checks before trusting the signed prekey. + * + * This is the whole reason a relay cannot quietly substitute its own prekey + * and read everything — provided the peer actually verifies it, which is + * why [E2ESession.initiate] does, unconditionally. + */ + fun signedPreKeySignature(): ByteArray = + Ed25519.sign(identitySigning.private, X25519.rawFromPublicKey(signedPreKey.public)) + + fun generateOneTimePreKeys(count: Int): List = (0 until count).map { + val id = nextPreKeyId++ + val pair = X25519.generate() + oneTimePreKeys[id] = pair + PreKey(id, X25519.rawFromPublicKey(pair.public)) + } + + /** Consume a one-time prekey. Consumed exactly once — that is the point of it. */ + fun takeOneTimePreKey(id: Int): KeyPair? = + if (id < 0) null else oneTimePreKeys.remove(id) + + val oneTimePreKeyCount: Int get() = oneTimePreKeys.size + + /** + * The signed prekey a prekey envelope refers to: the current one, or a + * retired one still inside the retention window. + * + * An id of 0 means the sender did not state one — true of every envelope + * built before rotation existed — and those can only have used the current + * key, so falling back is correct rather than lax. An id that IS stated and + * unknown returns null, because guessing would derive a garbage shared + * secret and report it as tampering. + */ + fun signedPreKeyFor(id: Int): KeyPair? = when { + id == 0 || id == signedPreKeyId -> signedPreKey + else -> retiredSignedPreKeys[id]?.keyPair + } + + fun shouldRotateSignedPreKey(now: Long = System.currentTimeMillis()): Boolean = + now - signedPreKeyCreatedAt >= ROTATE_AFTER_MS + + /** + * Retire the current signed prekey and mint a new one. + * + * ⚠️ The caller must republish the bundle and persist afterwards. Rotating + * without republishing leaves the relay handing out a bundle whose private + * half has moved; rotating without persisting loses the retired key on the + * next launch, which is the message-loss case above. + */ + fun rotateSignedPreKey(now: Long = System.currentTimeMillis()): KeyPair { + retiredSignedPreKeys[signedPreKeyId] = Retired(signedPreKey, now) + signedPreKey = X25519.generate() + signedPreKeyId += 1 + signedPreKeyCreatedAt = now + pruneSignedPreKeys(now) + return signedPreKey + } + + /** Drop retired prekeys no queued envelope can still refer to. */ + fun pruneSignedPreKeys(now: Long = System.currentTimeMillis()) { + retiredSignedPreKeys.entries.removeAll { now - it.value.retiredAt > RETAIN_AFTER_MS } + } + + /** The body of `POST /e2e/devices`. */ + fun registrationWire(oneTime: List = emptyList()): JSONObject = JSONObject() + .put("device_id", deviceId) + .put("ik_dh", Bytes.b64(identityDHPublic)) + .put("ik_sig", Bytes.b64(identitySigningPublic)) + .put("spk_id", signedPreKeyId) + .put("spk", Bytes.b64(X25519.rawFromPublicKey(signedPreKey.public))) + .put("spk_sig", Bytes.b64(signedPreKeySignature())) + .put("proto", 1) + .put("opks", org.json.JSONArray().also { array -> + oneTime.forEach { + array.put(JSONObject().put("id", it.id).put("pub", Bytes.b64(it.publicKey))) + } + }) + + /** + * Everything that must survive a restart. + * + * ⚠️ **This is private key material.** It is the vault's job to encrypt it; + * this returns it in the clear because a store that cannot see the plaintext + * cannot store it. Never log this, never write it to shared preferences. + * + * Retired prekeys travel with it, for the reason at the top of this file. + */ + fun archive(): JSONObject { + fun x(pair: KeyPair) = JSONObject() + .put("priv", Bytes.b64(pair.private.encoded)) + .put("pub", Bytes.b64(X25519.rawFromPublicKey(pair.public))) + + val otp = JSONObject() + oneTimePreKeys.forEach { (id, pair) -> otp.put(id.toString(), x(pair)) } + val previous = JSONObject() + retiredSignedPreKeys.forEach { (id, entry) -> + previous.put(id.toString(), x(entry.keyPair).put("retiredAt", entry.retiredAt)) + } + + return JSONObject() + .put("deviceId", deviceId) + .put("ikDH", x(identityDH)) + // Ed25519 is stored as its 32-byte seed rather than a PKCS#8 blob, + // because that is the one form both the JCA and a passphrase-derived + // account key agree on. + .put("ikSig", JSONObject() + .put("seed", Bytes.b64(Ed25519.seedOf(identitySigning.private))) + .put("pub", Bytes.b64(identitySigningPublic))) + .put("spk", x(signedPreKey)) + .put("spkId", signedPreKeyId) + .put("spkAt", signedPreKeyCreatedAt) + .put("nextId", nextPreKeyId) + .put("otp", otp) + .put("prevSpk", previous) + } +} diff --git a/engine/src/main/kotlin/io/flamenet/e2e/DoubleRatchet.kt b/engine/src/main/kotlin/io/flamenet/e2e/DoubleRatchet.kt index da336d9..8c2e8ac 100644 --- a/engine/src/main/kotlin/io/flamenet/e2e/DoubleRatchet.kt +++ b/engine/src/main/kotlin/io/flamenet/e2e/DoubleRatchet.kt @@ -1,5 +1,6 @@ package io.flamenet.e2e +import org.json.JSONObject import javax.crypto.Mac import javax.crypto.spec.SecretKeySpec @@ -84,6 +85,16 @@ class DoubleRatchet private constructor( */ const val MAX_SKIP = 1000 + /** + * How many skipped keys are kept across the whole session. + * + * `MAX_SKIP` bounds one header; without a second bound, a peer sending + * a thousand-message gap over and over grows this map forever. Oldest + * out first — a key that has waited through two thousand later messages + * is not arriving. + */ + const val MAX_STORE = 2000 + /** Matches `Aead`'s GCM nonce length; the ratchet strips it on the wire. */ private const val NONCE_LEN = 12 @@ -111,6 +122,35 @@ class DoubleRatchet private constructor( sent = 0, received = 0, previousChainLength = 0, ) + /** The other half of [archive]. */ + fun fromArchive(archive: JSONObject): DoubleRatchet { + fun opt(key: String) = archive.optString(key, "").takeIf { it.isNotEmpty() }?.let(Bytes::unb64) + val selfPrivate = opt("dhs") + val selfPublic = opt("dhsPub") + val ratchet = DoubleRatchet( + rootKey = Bytes.unb64(archive.getString("rk")), + sendingChain = opt("cks"), + receivingChain = opt("ckr"), + selfRatchet = if (selfPrivate != null && selfPublic != null) { + java.security.KeyPair( + X25519.publicKeyFromRaw(selfPublic), + X25519.privateKeyFromPkcs8(selfPrivate), + ) + } else null, + remoteRatchet = opt("dhr"), + sent = archive.getInt("ns"), + received = archive.getInt("nr"), + previousChainLength = archive.getInt("pn"), + ) + val skipped = archive.optJSONArray("skipped") ?: org.json.JSONArray() + for (i in 0 until skipped.length()) { + val entry = skipped.getJSONObject(i) + ratchet.skipped[SkippedKey(entry.getString("dh"), entry.getInt("n"))] = + Bytes.unb64(entry.getString("mk")) + } + return ratchet + } + private fun rootStep(rootKey: ByteArray, dhOutput: ByteArray): Pair { val out = Hkdf.derive(dhOutput, rootKey, RATCHET_INFO, 64) return out.copyOfRange(0, 32) to out.copyOfRange(32, 64) @@ -197,6 +237,9 @@ class DoubleRatchet private constructor( while (n < target) { val (mk, next) = chainStep(ck) skipped[SkippedKey(remote, n)] = mk + while (skipped.size > MAX_STORE) { + skipped.remove(skipped.keys.first()) + } ck = next n += 1 } @@ -224,4 +267,38 @@ class DoubleRatchet private constructor( rootKey = stepped.first sendingChain = stepped.second } + + /** + * Everything the conversation needs to continue after a restart. + * + * ⚠️ A ratchet that is not persisted is not a bug you notice — it is a bug + * your *contacts* notice. A client that loses session state falls back to + * minting a new device identity, and everyone who has ever talked to that + * account sees "security keys changed", which is the warning that is + * supposed to mean an attack. + * + * Skipped message keys travel too. They are the keys for messages that have + * not arrived yet, and dropping them at a restart loses exactly the + * messages that were already in flight. + */ + fun archive(): JSONObject { + val skippedJson = org.json.JSONArray() + skipped.forEach { (key, messageKey) -> + skippedJson.put(JSONObject() + .put("dh", key.ratchetPublic) + .put("n", key.index) + .put("mk", Bytes.b64(messageKey))) + } + return JSONObject() + .put("rk", Bytes.b64(rootKey)) + .put("cks", sendingChain?.let { Bytes.b64(it) }) + .put("ckr", receivingChain?.let { Bytes.b64(it) }) + .put("dhs", selfRatchet?.let { Bytes.b64(it.private.encoded) }) + .put("dhsPub", selfRatchet?.let { Bytes.b64(X25519.rawFromPublicKey(it.public)) }) + .put("dhr", remoteRatchet?.let { Bytes.b64(it) }) + .put("ns", sent) + .put("nr", received) + .put("pn", previousChainLength) + .put("skipped", skippedJson) + } } diff --git a/engine/src/main/kotlin/io/flamenet/e2e/E2ESession.kt b/engine/src/main/kotlin/io/flamenet/e2e/E2ESession.kt new file mode 100644 index 0000000..586e51e --- /dev/null +++ b/engine/src/main/kotlin/io/flamenet/e2e/E2ESession.kt @@ -0,0 +1,237 @@ +package io.flamenet.e2e + +import org.json.JSONObject + +/** + * One conversation with one device (spec §8.1). + * + * This is the layer an app actually uses: [initiate] or [respond] once, then + * [encrypt] and [decrypt] strings. Everything below it — the handshake, the + * ratchet, the associated data — is set up here and then never thought about + * again, which is the point. + * + * ⚠️ **The first messages are `prekey` envelopes and must stay that way until + * the peer replies.** Not until the first one is *sent* — until one is + * *answered*. Any single message may be the first to arrive, so all of them + * have to carry the handshake block; sending it once means losing that one + * message strands the whole conversation, and the symptom is a contact whose + * messages simply never appear. + * + * A port of `engine/src/e2e.js`, which is normative, and verified against it by + * `engine/test/interop-kotlin.mjs` rather than by reading. + */ +class E2ESession private constructor( + private val ratchet: DoubleRatchet, + /** Bound into every message's AEAD tag: the two identity keys, initiator first. */ + val associatedData: ByteArray, + private var pendingHandshake: JSONObject?, + val version: Int, +) { + + companion object { + + /** + * Start a conversation from a bundle fetched for a peer. + * + * ⚠️ **The signed prekey's signature is verified here, unconditionally.** + * The bundle arrives from a relay, and a relay that can substitute an + * unverified prekey can read the entire conversation. This is the single + * check that stops that, so it is not optional, not configurable, and + * not the caller's job to remember. + */ + fun initiate(identity: DeviceIdentity, bundle: PeerBundle): E2ESession { + if (!Ed25519.verify(bundle.identitySigning, bundle.signedPreKeySignature, bundle.signedPreKey)) { + throw SecurityException("the peer's signed prekey is not signed by its identity key") + } + + val initiated = X3DH.initiate( + identityPrivateDH = identity.identityDH.private, + bundle = X3DH.Bundle( + identityKeyDH = bundle.identityDH, + signedPreKey = bundle.signedPreKey, + oneTimePreKey = bundle.oneTimePreKey, + ), + ) + + val handshake = JSONObject() + .put("ik_dh", Bytes.b64(identity.identityDHPublic)) + .put("ik_sig", Bytes.b64(identity.identitySigningPublic)) + .put("ek", Bytes.b64(initiated.ephemeralPublic)) + .put("spk_id", bundle.signedPreKeyId) + .put("opk_id", bundle.oneTimePreKeyId) + + return E2ESession( + ratchet = DoubleRatchet.initiator(initiated.sharedSecret, bundle.signedPreKey), + associatedData = identity.identitySigningPublic + bundle.identitySigning, + pendingHandshake = handshake, + version = 1, + ) + } + + /** + * Answer a `prekey` envelope. + * + * ⚠️ The ratchet seeds from the signed prekey **X3DH actually used**, + * which may be a retired one. Seeding from the current key while X3DH + * used a retired one desynchronises the very first DH step, and the + * failure appears on the peer's screen, not here. + */ + fun respond(identity: DeviceIdentity, envelope: Envelope): E2ESession { + val handshake = envelope.handshake + ?: throw IllegalArgumentException("not a prekey envelope: no x3dh block") + + val signedPreKeyId = handshake.optInt("spk_id", 0) + val signedPreKey = identity.signedPreKeyFor(signedPreKeyId) + ?: throw IllegalArgumentException( + "unknown signed prekey $signedPreKeyId: retired past retention, or fabricated") + + val theirIdentityDH = Bytes.unb64(handshake.getString("ik_dh")) + val theirIdentitySigning = Bytes.unb64(handshake.getString("ik_sig")) + + val sharedSecret = X3DH.respond( + identityPrivateDH = identity.identityDH.private, + signedPreKeyPrivate = signedPreKey.private, + oneTimePrivate = identity.takeOneTimePreKey(handshake.optInt("opk_id", -1))?.private, + initiatorIdentityDH = theirIdentityDH, + initiatorEphemeral = Bytes.unb64(handshake.getString("ek")), + ) + + val session = E2ESession( + ratchet = DoubleRatchet.responder(sharedSecret, signedPreKey), + associatedData = theirIdentitySigning + identity.identitySigningPublic, + pendingHandshake = null, + version = 1, + ) + return session + } + + /** Restore a session written by [archive]. */ + fun fromArchive(archive: JSONObject): E2ESession = E2ESession( + ratchet = DoubleRatchet.fromArchive(archive.getJSONObject("ratchet")), + associatedData = Bytes.unb64(archive.getString("ad")), + pendingHandshake = archive.optJSONObject("pending"), + version = archive.optInt("version", 1), + ) + } + + fun encrypt(text: String): Envelope { + val message = ratchet.encrypt(text.toByteArray(Charsets.UTF_8), associatedData) + return Envelope( + version = version, + type = if (pendingHandshake != null) "prekey" else "msg", + header = message.header, + handshake = pendingHandshake, + ciphertext = message.ciphertext, + ) + } + + fun decrypt(envelope: Envelope): String { + val plaintext = ratchet.decrypt( + DoubleRatchet.Message(envelope.header, envelope.ciphertext), associatedData) + // Only now — a reply proves the peer has the handshake, which is the + // one moment it is safe to stop repeating it. + pendingHandshake = null + return String(plaintext, Charsets.UTF_8) + } + + /** + * ⚠️ [pendingHandshake] has to be archived too. It is the only copy of the + * ephemeral public key the handshake block contains, and regenerating it + * after a restart would produce a different block — a different session, + * from the peer's point of view. + */ + fun archive(): JSONObject = JSONObject() + .put("ratchet", ratchet.archive()) + .put("ad", Bytes.b64(associatedData)) + .put("pending", pendingHandshake) + .put("version", version) +} + +/** + * A peer's published keys, as fetched from `GET /e2e/keys`. + * + * ⚠️ `oneTimePreKey` being null is normal, not an error: the pool drains and + * the relay serves what is left. It costs initial forward secrecy for that one + * session and nothing else. + */ +data class PeerBundle( + val deviceId: String, + val identityDH: ByteArray, + val identitySigning: ByteArray, + val signedPreKeyId: Int, + val signedPreKey: ByteArray, + val signedPreKeySignature: ByteArray, + val oneTimePreKeyId: Int, + val oneTimePreKey: ByteArray?, + val proto: Int, +) { + companion object { + /** + * Read the relay's wire shape. + * + * ⚠️ The one bug this protocol has actually shipped was a client and a + * relay disagreeing about field names, and nothing caught it until a + * live round trip. So the translation lives in exactly one place, and + * that place is tested. + */ + fun fromWire(wire: JSONObject): PeerBundle { + val oneTime = wire.optJSONObject("opk") + return PeerBundle( + deviceId = wire.getString("device_id"), + identityDH = Bytes.unb64(wire.getString("ik_dh")), + identitySigning = Bytes.unb64(wire.getString("ik_sig")), + signedPreKeyId = wire.optInt("spk_id", 0), + signedPreKey = Bytes.unb64(wire.getString("spk")), + signedPreKeySignature = Bytes.unb64(wire.getString("spk_sig")), + oneTimePreKeyId = oneTime?.optInt("id", -1) ?: -1, + oneTimePreKey = oneTime?.let { Bytes.unb64(it.getString("pub")) }, + proto = wire.optInt("proto", 1), + ) + } + } +} + +/** One message on the wire (spec §8.2). */ +data class Envelope( + val version: Int, + val type: String, + val header: DoubleRatchet.Header, + val handshake: JSONObject?, + val ciphertext: ByteArray, +) { + fun toJson(): JSONObject = JSONObject() + .put("v", version) + .put("type", type) + .put("header", JSONObject() + .put("dh", Bytes.b64(header.dh)) + .put("pn", header.pn) + .put("n", header.n)) + .put("x3dh", handshake) + .put("ct", Bytes.b64(ciphertext)) + + /** + * The relay carries an envelope as one opaque base64 string, so it never + * sees these fields at all. + */ + fun toPayload(): String = Bytes.b64(toJson().toString().toByteArray(Charsets.UTF_8)) + + companion object { + fun fromJson(json: JSONObject): Envelope { + val header = json.getJSONObject("header") + return Envelope( + version = json.optInt("v", 1), + type = json.optString("type", "msg"), + header = DoubleRatchet.Header( + dh = Bytes.unb64(header.getString("dh")), + pn = header.getInt("pn"), + n = header.getInt("n"), + ), + handshake = json.optJSONObject("x3dh"), + ciphertext = Bytes.unb64(json.getString("ct")), + ) + } + + fun fromPayload(payload: String): Envelope = + fromJson(JSONObject(String(Bytes.unb64(payload), Charsets.UTF_8))) + } +} diff --git a/engine/src/main/kotlin/io/flamenet/e2e/Ed25519.kt b/engine/src/main/kotlin/io/flamenet/e2e/Ed25519.kt index 10b6958..4b1278e 100644 --- a/engine/src/main/kotlin/io/flamenet/e2e/Ed25519.kt +++ b/engine/src/main/kotlin/io/flamenet/e2e/Ed25519.kt @@ -124,8 +124,14 @@ object Ed25519 { private fun publicFromPrivate(privateKey: java.security.PrivateKey): java.security.PublicKey = publicKeyFromRaw(Rfc8032.publicKeyFromSeed(seedOf(privateKey))) - /** The 32 seed bytes back out of a PKCS#8 encoding. */ - private fun seedOf(privateKey: java.security.PrivateKey): ByteArray { + /** + * The 32 seed bytes back out of a PKCS#8 encoding. + * + * Public because the vault stores a seed, not a DER blob: it is the one + * representation the JCA, CryptoKit and a passphrase-derived account key + * all agree on. + */ + fun seedOf(privateKey: java.security.PrivateKey): ByteArray { val encoded = privateKey.encoded require(encoded.size == PKCS8_PREFIX.size + KEY_LEN) { "unexpected Ed25519 PKCS#8 length ${encoded.size}" diff --git a/engine/src/main/kotlin/io/flamenet/e2e/X25519.kt b/engine/src/main/kotlin/io/flamenet/e2e/X25519.kt index ec3d513..887b082 100644 --- a/engine/src/main/kotlin/io/flamenet/e2e/X25519.kt +++ b/engine/src/main/kotlin/io/flamenet/e2e/X25519.kt @@ -3,6 +3,7 @@ package io.flamenet.e2e import java.security.KeyFactory import java.security.KeyPair import java.security.KeyPairGenerator +import java.security.spec.PKCS8EncodedKeySpec import java.security.spec.X509EncodedKeySpec import javax.crypto.KeyAgreement @@ -52,6 +53,18 @@ object X25519 { .generatePublic(X509EncodedKeySpec(SPKI_PREFIX + raw)) } + /** + * A private key back from its PKCS#8 encoding, which is what the vault + * stores. + * + * PKCS#8 rather than the raw 32 scalar bytes for the same reason public + * keys go through DER: Conscrypt will not build an XDH key from raw bytes. + * Storing raw and wrapping at load time would work on a laptop and throw on + * a phone, which is the exact bug this engine has already had once. + */ + fun privateKeyFromPkcs8(der: ByteArray): java.security.PrivateKey = + KeyFactory.getInstance("XDH").generatePrivate(PKCS8EncodedKeySpec(der)) + /** The raw 32 bytes the wire carries, from a JCA public key. */ fun rawFromPublicKey(key: java.security.PublicKey): ByteArray { val encoded = key.encoded diff --git a/engine/src/test/kotlin/io/flamenet/e2e/E2ESessionTest.kt b/engine/src/test/kotlin/io/flamenet/e2e/E2ESessionTest.kt new file mode 100644 index 0000000..9aeb20b --- /dev/null +++ b/engine/src/test/kotlin/io/flamenet/e2e/E2ESessionTest.kt @@ -0,0 +1,211 @@ +package io.flamenet.e2e + +import org.json.JSONObject +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertNotEquals +import kotlin.test.assertNull +import kotlin.test.assertTrue + +class E2ESessionTest { + + /** A bundle for [identity], as its relay would serve one. */ + private fun bundleFor(identity: DeviceIdentity, withOneTime: Boolean = true): PeerBundle { + val oneTime = if (withOneTime) identity.generateOneTimePreKeys(1).first() else null + return PeerBundle.fromWire(identity.registrationWire().also { wire -> + if (oneTime != null) { + wire.put("opk", JSONObject() + .put("id", oneTime.id) + .put("pub", Bytes.b64(oneTime.publicKey))) + } + }) + } + + @Test + fun `a conversation runs both ways`() { + val alice = DeviceIdentity.create() + val bob = DeviceIdentity.create() + + val a = E2ESession.initiate(alice, bundleFor(bob)) + val first = a.encrypt("hello") + val b = E2ESession.respond(bob, first) + assertEquals("hello", b.decrypt(first)) + + assertEquals("hi back", a.decrypt(b.encrypt("hi back"))) + assertEquals("and again", b.decrypt(a.encrypt("and again"))) + } + + /** + * ⚠️ The handshake has to ride EVERY message until one is answered, not + * just the first. Any message may be the one that arrives first, and a + * conversation where the peer never received the handshake looks, from the + * outside, like a contact who has stopped replying. + */ + @Test + fun `the handshake repeats until the peer answers`() { + val alice = DeviceIdentity.create() + val bob = DeviceIdentity.create() + val a = E2ESession.initiate(alice, bundleFor(bob)) + + val queued = (1..3).map { a.encrypt("m$it") } + assertTrue(queued.all { it.type == "prekey" && it.handshake != null }) + + // Bob receives the THIRD one first. It must be enough on its own. + val b = E2ESession.respond(bob, queued[2]) + assertEquals("m3", b.decrypt(queued[2])) + assertEquals("m1", b.decrypt(queued[0])) + assertEquals("m2", b.decrypt(queued[1])) + + // Once Bob replies and Alice reads it, the block stops. + a.decrypt(b.encrypt("got them")) + assertEquals("msg", a.encrypt("now plain").type) + assertNull(a.encrypt("still plain").handshake) + } + + /** + * The bug this exists to stop: sessions that lived only in memory. A client + * that loses them re-registers a device, and every contact is shown + * "security keys changed" — the warning that is supposed to mean an attack. + */ + @Test + fun `a session survives being archived mid-conversation`() { + val alice = DeviceIdentity.create() + val bob = DeviceIdentity.create() + + var a = E2ESession.initiate(alice, bundleFor(bob)) + val first = a.encrypt("before the restart") + val b = E2ESession.respond(bob, first) + assertEquals("before the restart", b.decrypt(first)) + + // Both sides go through a restart, at different points in the chain. + a = E2ESession.fromArchive(JSONObject(a.archive().toString())) + val restoredB = E2ESession.fromArchive(JSONObject(b.archive().toString())) + + assertEquals("after the restart", restoredB.decrypt(a.encrypt("after the restart"))) + assertEquals("and back", a.decrypt(restoredB.encrypt("and back"))) + } + + /** Skipped keys are the in-flight messages; losing them at a restart loses those. */ + @Test + fun `messages in flight across a restart still open`() { + val alice = DeviceIdentity.create() + val bob = DeviceIdentity.create() + val a = E2ESession.initiate(alice, bundleFor(bob)) + val first = a.encrypt("m0") + var b = E2ESession.respond(bob, first) + b.decrypt(first) + + val later = (1..3).map { a.encrypt("m$it") } + b.decrypt(later[2]) // arrives early; 1 and 2 are now skipped keys + b = E2ESession.fromArchive(JSONObject(b.archive().toString())) + assertEquals("m1", b.decrypt(later[0])) + assertEquals("m2", b.decrypt(later[1])) + } + + @Test + fun `an unsigned prekey is refused`() { + val alice = DeviceIdentity.create() + val bob = DeviceIdentity.create() + val impostor = DeviceIdentity.create() + + // A relay substituting its own prekey, signed by a key it controls. + // Nothing else in the protocol catches this. + val tampered = bundleFor(bob).copy( + signedPreKey = X25519.rawFromPublicKey(impostor.signedPreKey.public), + signedPreKeySignature = impostor.signedPreKeySignature(), + ) + assertFailsWith { E2ESession.initiate(alice, tampered) } + } + + @Test + fun `a session opens when the one-time prekey pool is drained`() { + val alice = DeviceIdentity.create() + val bob = DeviceIdentity.create() + val a = E2ESession.initiate(alice, bundleFor(bob, withOneTime = false)) + val first = a.encrypt("no one-time prekey") + val b = E2ESession.respond(bob, first) + assertEquals("no one-time prekey", b.decrypt(first)) + } + + /** + * Rotation without retention is message loss. A sender that fetched a + * bundle before the rotation is still entitled to be understood. + */ + @Test + fun `an envelope built against a retired signed prekey still opens`() { + val alice = DeviceIdentity.create() + val bob = DeviceIdentity.create() + + val old = bundleFor(bob) + val a = E2ESession.initiate(alice, old) + val queued = a.encrypt("sent before the rotation") + + bob.rotateSignedPreKey() + assertNotEquals(old.signedPreKeyId, bob.signedPreKeyId) + + val b = E2ESession.respond(bob, queued) + assertEquals("sent before the rotation", b.decrypt(queued)) + } + + @Test + fun `a signed prekey retired past retention is refused rather than guessed`() { + val alice = DeviceIdentity.create() + val bob = DeviceIdentity.create() + val queued = E2ESession.initiate(alice, bundleFor(bob)).encrypt("far too late") + + // Rotate, then let the retention window pass. Guessing at the current + // key here would derive a garbage secret and report it as tampering. + bob.rotateSignedPreKey() + bob.pruneSignedPreKeys(System.currentTimeMillis() + DeviceIdentity.RETAIN_AFTER_MS + 1) + assertFailsWith { E2ESession.respond(bob, queued) } + } + + @Test + fun `a one-time prekey is consumed exactly once`() { + val bob = DeviceIdentity.create() + val key = bob.generateOneTimePreKeys(1).first() + assertEquals(1, bob.oneTimePreKeyCount) + assertTrue(bob.takeOneTimePreKey(key.id) != null) + assertNull(bob.takeOneTimePreKey(key.id)) + assertEquals(0, bob.oneTimePreKeyCount) + } + + @Test + fun `an envelope survives the payload round trip`() { + val alice = DeviceIdentity.create() + val bob = DeviceIdentity.create() + val a = E2ESession.initiate(alice, bundleFor(bob)) + val sent = a.encrypt("through the relay 👋") + + val received = Envelope.fromPayload(sent.toPayload()) + assertEquals("prekey", received.type) + val b = E2ESession.respond(bob, received) + assertEquals("through the relay 👋", b.decrypt(received)) + } + + @Test + fun `a device identity survives being archived`() { + val original = DeviceIdentity.create() + original.generateOneTimePreKeys(3) + original.rotateSignedPreKey() + + val restored = DeviceIdentity.fromArchive(JSONObject(original.archive().toString())) + assertEquals(original.deviceId, restored.deviceId) + assertEquals(original.signedPreKeyId, restored.signedPreKeyId) + assertEquals(3, restored.oneTimePreKeyCount) + assertContentEqualsB64(original.identityDHPublic, restored.identityDHPublic) + assertContentEqualsB64(original.identitySigningPublic, restored.identitySigningPublic) + + // The restored device must still be able to complete a handshake, which + // is the only thing that proves the PRIVATE halves came back too. + val alice = DeviceIdentity.create() + val bundle = PeerBundle.fromWire(restored.registrationWire()) + val session = E2ESession.initiate(alice, bundle) + val first = session.encrypt("after a reload") + assertEquals("after a reload", E2ESession.respond(restored, first).decrypt(first)) + } + + private fun assertContentEqualsB64(expected: ByteArray, actual: ByteArray) = + assertEquals(Bytes.b64(expected), Bytes.b64(actual)) +} diff --git a/interop/src/main/kotlin/io/flamenet/interop/Main.kt b/interop/src/main/kotlin/io/flamenet/interop/Main.kt index 326f234..620f21b 100644 --- a/interop/src/main/kotlin/io/flamenet/interop/Main.kt +++ b/interop/src/main/kotlin/io/flamenet/interop/Main.kt @@ -2,7 +2,11 @@ package io.flamenet.interop import io.flamenet.e2e.AccountKdf import io.flamenet.e2e.Aead +import io.flamenet.e2e.DeviceIdentity import io.flamenet.e2e.DoubleRatchet +import io.flamenet.e2e.E2ESession +import io.flamenet.e2e.Envelope +import io.flamenet.e2e.PeerBundle import io.flamenet.e2e.Ed25519 import io.flamenet.e2e.Hkdf import io.flamenet.e2e.KeyLog @@ -86,6 +90,13 @@ private fun handle(req: JSONObject): JSONObject = when (req.getString("cmd")) { "respond" -> respond(req) "encrypt" -> encrypt(req) "decrypt" -> decrypt(req) + // --- the session layer, which is what an app actually uses ------------ + "session_device" -> sessionDevice() + "session_initiate" -> sessionInitiate(req) + "session_respond" -> sessionRespond(req) + "session_encrypt" -> sessionEncrypt(req) + "session_decrypt" -> sessionDecrypt(req) + "session_reload" -> sessionReload(req) "safety" -> JSONObject().put( "number", SafetyNumber.compute(req.bytes("local"), req.bytes("remote"))) "account_kdf" -> accountKdf(req) @@ -237,6 +248,73 @@ private fun messageFrom(req: JSONObject): DoubleRatchet.Message { ) } +// --------------------------------------------------------------------------- +// Session layer +// --------------------------------------------------------------------------- + +private val identities = HashMap() +private val liveSessions = HashMap() + +/** A device and the bundle wire a relay would serve for it, one prekey included. */ +private fun sessionDevice(): JSONObject { + val identity = DeviceIdentity.create() + identities[identity.deviceId] = identity + val oneTime = identity.generateOneTimePreKeys(1).first() + val wire = identity.registrationWire() + .put("opk", JSONObject().put("id", oneTime.id).put("pub", b64(oneTime.publicKey))) + return JSONObject().put("device", identity.deviceId).put("bundle", wire) +} + +private fun sessionInitiate(req: JSONObject): JSONObject { + val identity = identities.getValue(req.getString("device")) + val session = E2ESession.initiate(identity, PeerBundle.fromWire(req.getJSONObject("bundle"))) + val id = next("kotlin-e2e-") + liveSessions[id] = session + return JSONObject() + .put("session", id) + .put("ad", b64(session.associatedData)) + // The full envelope, exactly as it would go on the wire. This is the + // layer where the one bug this protocol has actually shipped lives: a + // client and a relay disagreeing about a field name, caught by nothing + // until a live round trip. + .put("envelope", session.encrypt(req.getString("text")).toJson()) +} + +private fun sessionRespond(req: JSONObject): JSONObject { + val identity = identities.getValue(req.getString("device")) + val envelope = Envelope.fromJson(req.getJSONObject("envelope")) + val session = E2ESession.respond(identity, envelope) + val id = next("kotlin-e2e-") + liveSessions[id] = session + return JSONObject() + .put("session", id) + .put("ad", b64(session.associatedData)) + .put("text", session.decrypt(envelope)) +} + +private fun sessionEncrypt(req: JSONObject): JSONObject = JSONObject() + .put("envelope", liveSessions.getValue(req.getString("session")) + .encrypt(req.getString("text")).toJson()) + +private fun sessionDecrypt(req: JSONObject): JSONObject = JSONObject() + .put("text", liveSessions.getValue(req.getString("session")) + .decrypt(Envelope.fromJson(req.getJSONObject("envelope")))) + +/** + * Archive a session and restore it, in place. + * + * Exposed to the harness so the persistence path is exercised by a real + * cross-engine conversation rather than only by a Kotlin test talking to + * itself. A session that survives a restart against our own bytes but not + * against the peer's is a distinction the user experiences. + */ +private fun sessionReload(req: JSONObject): JSONObject { + val id = req.getString("session") + val archived = liveSessions.getValue(id).archive() + liveSessions[id] = E2ESession.fromArchive(JSONObject(archived.toString())) + return JSONObject().put("reloaded", true).put("bytes", archived.toString().length) +} + private fun accountKdf(req: JSONObject): JSONObject { val passphrase = req.getString("passphrase") val salt = req.bytes("salt")