Kotlin: the Double Ratchet
The second half of a protocol Android did not have. With X3DH it can now agree a secret, and with this it can hold a conversation. Two ratchets: the symmetric one advances per message so a compromised key reveals nothing earlier, the DH one advances on each direction change so a conversation recovers from compromise rather than staying broken. The tests run conversations rather than checking steps, because that is what catches the bugs this code actually has. X3DH is a pure function — it agrees or it does not. A ratchet can agree for six messages and diverge on the seventh because one side counted a skipped message differently, and the symptom is only that something will not decrypt. So: out-of-order delivery, a message lost forever, a gap ACROSS a direction change (which is the whole reason "pn" is on the wire), and four DH turns in sequence. Three things that would each produce plausible-but-wrong bytes, so each is tested rather than reasoned about: - the sending chain advances BEFORE the key is used, so a caller that retries cannot encrypt twice under one key — two GCM encryptions under one key and nonce leak the XOR of both plaintexts - the header's canonical JSON is hand-built, key order "dh, pn, n", no spaces, because it goes into the AAD and three implementations must reproduce it byte-for-byte - the derived nonce is stripped from the wire rather than sent, so a receiver cannot be handed a nonce that disagrees with the one its key implies MAX_SKIP caps derivation at 1000: a header claiming a huge counter costs the sender nothing and the receiver everything. 13 tests. Still missing before an app: a session layer, a wire client, and the app itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Paul Hitt
co-authored by
Claude Opus 5
Claude Opus 5
parent
c893d412d2
commit
e0589eff14
@@ -0,0 +1,227 @@
|
||||
package io.flamenet.e2e
|
||||
|
||||
import javax.crypto.Mac
|
||||
import javax.crypto.spec.SecretKeySpec
|
||||
|
||||
/**
|
||||
* The Double Ratchet (spec §6), with unencrypted headers.
|
||||
*
|
||||
* Two ratchets turning at different rates. The **symmetric** one advances once
|
||||
* per message, so a compromised message key reveals nothing about earlier ones.
|
||||
* The **Diffie-Hellman** one advances whenever the other side sends a new
|
||||
* ratchet public key, which is what lets a conversation recover after a
|
||||
* compromise rather than staying broken forever.
|
||||
*
|
||||
* ```
|
||||
* MK = HMAC-SHA256(key = CK, msg = 0x01)
|
||||
* CK' = HMAC-SHA256(key = CK, msg = 0x02)
|
||||
* (RK', CK') = HKDF-SHA256(IKM = dh_out, salt = RK,
|
||||
* info = "FlamenetE2E_Ratchet_v1", L = 64)
|
||||
* ```
|
||||
*
|
||||
* ⚠️ **Everything here is state, and state is where this goes wrong.** X3DH is
|
||||
* a pure function and either agrees or does not. A ratchet can agree for six
|
||||
* messages and then diverge on the seventh because one side counted a skipped
|
||||
* message differently — and the symptom, again, is only that something will not
|
||||
* decrypt. The tests below therefore run conversations, out of order and with
|
||||
* gaps, rather than checking any single step.
|
||||
*
|
||||
* ⚠️ **Never encrypt twice under one message key.** Two AES-GCM encryptions
|
||||
* under the same key and nonce leak the XOR of both plaintexts and destroy the
|
||||
* authentication guarantee — it is a catastrophic failure, not a degraded one.
|
||||
* The sending chain advances before the key is used, so a key cannot be reused
|
||||
* even if a caller retries.
|
||||
*
|
||||
* A port of `engine/src/e2e.js`, which is normative.
|
||||
*/
|
||||
class DoubleRatchet private constructor(
|
||||
private var rootKey: ByteArray,
|
||||
private var sendingChain: ByteArray?,
|
||||
private var receivingChain: ByteArray?,
|
||||
private var selfRatchet: java.security.KeyPair?,
|
||||
private var remoteRatchet: ByteArray?,
|
||||
/** Messages sent in the current sending chain. */
|
||||
private var sent: Int,
|
||||
/** Messages received in the current receiving chain. */
|
||||
private var received: Int,
|
||||
/** Length of the PREVIOUS sending chain, so the far side knows how many to skip. */
|
||||
private var previousChainLength: Int,
|
||||
) {
|
||||
|
||||
/** Keys for messages that arrived out of order, or have not arrived yet. */
|
||||
private val skipped = LinkedHashMap<SkippedKey, ByteArray>()
|
||||
|
||||
data class SkippedKey(val ratchetPublic: String, val index: Int)
|
||||
|
||||
data class Header(val dh: ByteArray, val pn: Int, val n: Int) {
|
||||
/**
|
||||
* The exact bytes that go into the AAD.
|
||||
*
|
||||
* Hand-built rather than serialised by a JSON library: the AAD has to be
|
||||
* reproduced byte-for-byte by three implementations, and an encoder is
|
||||
* free to reorder keys or add spaces between releases. Key order is
|
||||
* `dh, pn, n` and there are no spaces — both are part of the protocol.
|
||||
*/
|
||||
fun canonicalJson(): ByteArray =
|
||||
("{\"dh\":\"" + Bytes.b64(dh) + "\",\"pn\":" + pn + ",\"n\":" + n + "}")
|
||||
.toByteArray(Charsets.UTF_8)
|
||||
}
|
||||
|
||||
data class Message(val header: Header, val ciphertext: ByteArray)
|
||||
|
||||
companion object {
|
||||
private val RATCHET_INFO = "FlamenetE2E_Ratchet_v1".toByteArray(Charsets.UTF_8)
|
||||
private val MSGKEY_INFO = "FlamenetE2E_MsgKey_v1".toByteArray(Charsets.UTF_8)
|
||||
private val ZERO_SALT = ByteArray(32)
|
||||
|
||||
/**
|
||||
* A cap on how many keys a single header may make us derive.
|
||||
*
|
||||
* Without one, a header claiming `n = 2_000_000_000` is a denial of
|
||||
* service that costs the sender nothing: the receiver dutifully derives
|
||||
* two billion keys. The number is generous for real reordering and
|
||||
* ruinous for that.
|
||||
*/
|
||||
const val MAX_SKIP = 1000
|
||||
|
||||
/** Matches `Aead`'s GCM nonce length; the ratchet strips it on the wire. */
|
||||
private const val NONCE_LEN = 12
|
||||
|
||||
/** Alice, who has just completed X3DH and knows Bob's signed prekey. */
|
||||
fun initiator(sharedSecret: ByteArray, remoteSignedPreKey: ByteArray): DoubleRatchet {
|
||||
val self = X25519.generate()
|
||||
val (rk, ck) = rootStep(sharedSecret, X25519.agree(self.private, remoteSignedPreKey))
|
||||
return DoubleRatchet(
|
||||
rootKey = rk, sendingChain = ck, receivingChain = null,
|
||||
selfRatchet = self, remoteRatchet = remoteSignedPreKey,
|
||||
sent = 0, received = 0, previousChainLength = 0,
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Bob, whose signed prekey IS his first ratchet key.
|
||||
*
|
||||
* He does not step the root yet: he has nothing to step it with until a
|
||||
* message from Alice tells him her ratchet public key.
|
||||
*/
|
||||
fun responder(sharedSecret: ByteArray, signedPreKey: java.security.KeyPair): DoubleRatchet =
|
||||
DoubleRatchet(
|
||||
rootKey = sharedSecret, sendingChain = null, receivingChain = null,
|
||||
selfRatchet = signedPreKey, remoteRatchet = null,
|
||||
sent = 0, received = 0, previousChainLength = 0,
|
||||
)
|
||||
|
||||
private fun rootStep(rootKey: ByteArray, dhOutput: ByteArray): Pair<ByteArray, ByteArray> {
|
||||
val out = Hkdf.derive(dhOutput, rootKey, RATCHET_INFO, 64)
|
||||
return out.copyOfRange(0, 32) to out.copyOfRange(32, 64)
|
||||
}
|
||||
|
||||
private fun hmac(key: ByteArray, msg: ByteArray): ByteArray {
|
||||
val mac = Mac.getInstance("HmacSHA256")
|
||||
mac.init(SecretKeySpec(key, "HmacSHA256"))
|
||||
return mac.doFinal(msg)
|
||||
}
|
||||
|
||||
/** One symmetric step: a message key, and the chain key that replaces it. */
|
||||
private fun chainStep(chainKey: ByteArray): Pair<ByteArray, ByteArray> =
|
||||
hmac(chainKey, byteArrayOf(0x01)) to hmac(chainKey, byteArrayOf(0x02))
|
||||
|
||||
/** A message key expands into an AES key and the nonce it is used with. */
|
||||
private fun aeadParams(messageKey: ByteArray): Pair<ByteArray, ByteArray> {
|
||||
val okm = Hkdf.derive(messageKey, ZERO_SALT, MSGKEY_INFO, 44)
|
||||
return okm.copyOfRange(0, 32) to okm.copyOfRange(32, 44)
|
||||
}
|
||||
}
|
||||
|
||||
fun encrypt(plaintext: ByteArray, associatedData: ByteArray): Message {
|
||||
val chain = sendingChain ?: error("cannot send before the first ratchet step")
|
||||
val (messageKey, nextChain) = chainStep(chain)
|
||||
// Advanced BEFORE use, so a caller that retries cannot encrypt twice
|
||||
// under one key.
|
||||
sendingChain = nextChain
|
||||
|
||||
val header = Header(
|
||||
dh = X25519.rawFromPublicKey(selfRatchet!!.public),
|
||||
pn = previousChainLength,
|
||||
n = sent,
|
||||
)
|
||||
sent += 1
|
||||
|
||||
val (aesKey, nonce) = aeadParams(messageKey)
|
||||
// `Aead.seal` returns `nonce || ct || tag`, and the nonce is dropped
|
||||
// here: the ratchet DERIVES it from the message key, so transmitting it
|
||||
// would put twelve redundant bytes on every message and invite a
|
||||
// receiver to trust a nonce off the wire instead of the one the key
|
||||
// says it must be.
|
||||
val sealed = Aead.seal(aesKey, nonce, plaintext, associatedData + header.canonicalJson())
|
||||
return Message(header, sealed.copyOfRange(NONCE_LEN, sealed.size))
|
||||
}
|
||||
|
||||
fun decrypt(message: Message, associatedData: ByteArray): ByteArray {
|
||||
val aad = associatedData + message.header.canonicalJson()
|
||||
|
||||
// A key held for a message that never arrived, or arrived late.
|
||||
val skippedKey = SkippedKey(Bytes.b64(message.header.dh), message.header.n)
|
||||
skipped.remove(skippedKey)?.let { mk ->
|
||||
val (aesKey, nonce) = aeadParams(mk)
|
||||
return Aead.open(aesKey, nonce + message.ciphertext, aad)
|
||||
}
|
||||
|
||||
// A new ratchet key from the far side: catch up on their old chain,
|
||||
// then turn the DH ratchet.
|
||||
if (remoteRatchet == null || !message.header.dh.contentEquals(remoteRatchet)) {
|
||||
skipTo(message.header.pn)
|
||||
dhRatchet(message.header.dh)
|
||||
}
|
||||
skipTo(message.header.n)
|
||||
|
||||
val chain = receivingChain ?: error("no receiving chain")
|
||||
val (messageKey, nextChain) = chainStep(chain)
|
||||
receivingChain = nextChain
|
||||
received += 1
|
||||
|
||||
val (aesKey, nonce) = aeadParams(messageKey)
|
||||
return Aead.open(aesKey, nonce + message.ciphertext, aad)
|
||||
}
|
||||
|
||||
/** Derive and store keys for messages we have not seen yet. */
|
||||
private fun skipTo(target: Int) {
|
||||
val chain = receivingChain ?: return
|
||||
if (target < received) return
|
||||
require(target - received <= MAX_SKIP) {
|
||||
"refusing to derive ${target - received} skipped keys"
|
||||
}
|
||||
var ck = chain
|
||||
var n = received
|
||||
val remote = Bytes.b64(remoteRatchet ?: return)
|
||||
while (n < target) {
|
||||
val (mk, next) = chainStep(ck)
|
||||
skipped[SkippedKey(remote, n)] = mk
|
||||
ck = next
|
||||
n += 1
|
||||
}
|
||||
receivingChain = ck
|
||||
received = n
|
||||
}
|
||||
|
||||
/**
|
||||
* Turn the DH ratchet: one root step to open a receiving chain against
|
||||
* their new key, a fresh keypair of our own, and a second root step to open
|
||||
* the sending chain they will decrypt with.
|
||||
*/
|
||||
private fun dhRatchet(theirRatchet: ByteArray) {
|
||||
previousChainLength = sent
|
||||
sent = 0
|
||||
received = 0
|
||||
remoteRatchet = theirRatchet
|
||||
|
||||
var (rk, ck) = rootStep(rootKey, X25519.agree(selfRatchet!!.private, theirRatchet))
|
||||
rootKey = rk
|
||||
receivingChain = ck
|
||||
|
||||
selfRatchet = X25519.generate()
|
||||
val stepped = rootStep(rootKey, X25519.agree(selfRatchet!!.private, theirRatchet))
|
||||
rootKey = stepped.first
|
||||
sendingChain = stepped.second
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,148 @@
|
||||
package io.flamenet.e2e
|
||||
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertContentEquals
|
||||
import kotlin.test.assertEquals
|
||||
import kotlin.test.assertFailsWith
|
||||
import kotlin.test.assertTrue
|
||||
|
||||
/**
|
||||
* The ratchet, driven as a conversation.
|
||||
*
|
||||
* Single-step assertions would pass on a ratchet that is subtly wrong. What
|
||||
* catches a counting error is a conversation that goes back and forth, drops
|
||||
* messages, and delivers some out of order — because the divergence only shows
|
||||
* up once the two sides have taken a different number of steps.
|
||||
*/
|
||||
class DoubleRatchetTest {
|
||||
|
||||
private val AD = "flamenet-test-ad".toByteArray()
|
||||
|
||||
/** Alice and Bob, having already agreed a secret via X3DH. */
|
||||
private fun pair(): Pair<DoubleRatchet, DoubleRatchet> {
|
||||
val bobSigned = X25519.generate()
|
||||
val aliceIdentity = X25519.generate()
|
||||
val bobIdentity = X25519.generate()
|
||||
|
||||
val initiated = X3DH.initiate(
|
||||
aliceIdentity.private,
|
||||
X3DH.Bundle(
|
||||
identityKeyDH = X25519.rawFromPublicKey(bobIdentity.public),
|
||||
signedPreKey = X25519.rawFromPublicKey(bobSigned.public),
|
||||
oneTimePreKey = null,
|
||||
),
|
||||
)
|
||||
val bobSecret = X3DH.respond(
|
||||
identityPrivateDH = bobIdentity.private,
|
||||
signedPreKeyPrivate = bobSigned.private,
|
||||
oneTimePrivate = null,
|
||||
initiatorIdentityDH = X25519.rawFromPublicKey(aliceIdentity.public),
|
||||
initiatorEphemeral = initiated.ephemeralPublic,
|
||||
)
|
||||
assertContentEquals(initiated.sharedSecret, bobSecret, "X3DH disagreed before the ratchet started")
|
||||
|
||||
val alice = DoubleRatchet.initiator(
|
||||
initiated.sharedSecret, X25519.rawFromPublicKey(bobSigned.public))
|
||||
val bob = DoubleRatchet.responder(bobSecret, bobSigned)
|
||||
return alice to bob
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `a message survives the trip`() {
|
||||
val (alice, bob) = pair()
|
||||
val msg = alice.encrypt("hello".toByteArray(), AD)
|
||||
assertContentEquals("hello".toByteArray(), bob.decrypt(msg, AD))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `a conversation going both ways stays in step`() {
|
||||
// Each direction change turns the DH ratchet. Four turns is enough for a
|
||||
// root-key or chain-key mistake to compound into a failure.
|
||||
val (alice, bob) = pair()
|
||||
assertContentEquals("a1".toByteArray(), bob.decrypt(alice.encrypt("a1".toByteArray(), AD), AD))
|
||||
assertContentEquals("b1".toByteArray(), alice.decrypt(bob.encrypt("b1".toByteArray(), AD), AD))
|
||||
assertContentEquals("a2".toByteArray(), bob.decrypt(alice.encrypt("a2".toByteArray(), AD), AD))
|
||||
assertContentEquals("b2".toByteArray(), alice.decrypt(bob.encrypt("b2".toByteArray(), AD), AD))
|
||||
assertContentEquals("a3".toByteArray(), bob.decrypt(alice.encrypt("a3".toByteArray(), AD), AD))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `messages arriving out of order still open`() {
|
||||
val (alice, bob) = pair()
|
||||
val one = alice.encrypt("one".toByteArray(), AD)
|
||||
val two = alice.encrypt("two".toByteArray(), AD)
|
||||
val three = alice.encrypt("three".toByteArray(), AD)
|
||||
|
||||
// Delivered 3, 1, 2 — the receiver must hold keys for the gap rather
|
||||
// than advancing past them.
|
||||
assertContentEquals("three".toByteArray(), bob.decrypt(three, AD))
|
||||
assertContentEquals("one".toByteArray(), bob.decrypt(one, AD))
|
||||
assertContentEquals("two".toByteArray(), bob.decrypt(two, AD))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `a message lost forever does not break the ones after it`() {
|
||||
val (alice, bob) = pair()
|
||||
alice.encrypt("lost".toByteArray(), AD) // never delivered
|
||||
val after = alice.encrypt("after".toByteArray(), AD)
|
||||
assertContentEquals("after".toByteArray(), bob.decrypt(after, AD))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `a gap across a direction change is handled`() {
|
||||
// `pn` exists for exactly this: Bob has to know how many messages were
|
||||
// in Alice's previous chain to skip them when she starts a new one.
|
||||
val (alice, bob) = pair()
|
||||
assertContentEquals("a1".toByteArray(), bob.decrypt(alice.encrypt("a1".toByteArray(), AD), AD))
|
||||
assertContentEquals("b1".toByteArray(), alice.decrypt(bob.encrypt("b1".toByteArray(), AD), AD))
|
||||
|
||||
alice.encrypt("dropped".toByteArray(), AD) // never delivered
|
||||
val next = alice.encrypt("next".toByteArray(), AD)
|
||||
assertContentEquals("next".toByteArray(), bob.decrypt(next, AD))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `the header is bound to the ciphertext`() {
|
||||
// The AAD covers the canonical header, so editing the counter must break
|
||||
// the tag rather than silently addressing a different message key.
|
||||
val (alice, bob) = pair()
|
||||
val msg = alice.encrypt("bound".toByteArray(), AD)
|
||||
val tampered = DoubleRatchet.Message(
|
||||
DoubleRatchet.Header(msg.header.dh, msg.header.pn, msg.header.n + 5),
|
||||
msg.ciphertext,
|
||||
)
|
||||
assertFailsWith<Throwable> { bob.decrypt(tampered, AD) }
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `a different associated data does not open the message`() {
|
||||
val (alice, bob) = pair()
|
||||
val msg = alice.encrypt("bound".toByteArray(), AD)
|
||||
assertFailsWith<Throwable> { bob.decrypt(msg, "other-ad".toByteArray()) }
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `an absurd skip is refused rather than computed`() {
|
||||
// A header claiming a huge counter costs the sender nothing and would
|
||||
// cost the receiver everything.
|
||||
val (alice, bob) = pair()
|
||||
val msg = alice.encrypt("x".toByteArray(), AD)
|
||||
val absurd = DoubleRatchet.Message(
|
||||
DoubleRatchet.Header(msg.header.dh, msg.header.pn, 500_000),
|
||||
msg.ciphertext,
|
||||
)
|
||||
assertFailsWith<IllegalArgumentException> { bob.decrypt(absurd, AD) }
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `every message uses a different key`() {
|
||||
// If two messages ever produced identical ciphertext for identical
|
||||
// plaintext, the chain would not be advancing.
|
||||
val (alice, _) = pair()
|
||||
val first = alice.encrypt("same".toByteArray(), AD)
|
||||
val second = alice.encrypt("same".toByteArray(), AD)
|
||||
assertTrue(!first.ciphertext.contentEquals(second.ciphertext))
|
||||
assertEquals(0, first.header.n)
|
||||
assertEquals(1, second.header.n)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user