obfs4 for Android, part 3: SipHash, the length DRBG, and ntor

64/64 handshakes agree with lyrebird on KEY_SEED, AUTH and KDF output,
from both the client and the server side. 23 tests.

SipHash is the one primitive hand-written here despite BouncyCastle
shipping one, and that is worth justifying because everything else in
this module is BouncyCastle's. BC's SipHash is a Mac: doFinal RESETS.
obfs4's DRBG writes eight bytes, finalises, writes eight more and
finalises again -- without ever resetting. Emulating that with a
resetting MAC means re-hashing the whole history per block, O(n^2) in the
frames on a connection. So the state machine is written out, and `sum`
finalises a COPY.

The DRBG is checked ten blocks deep, not one. obfs4's comment says each
block is a fresh H(K, previous); the code never resets, so blocks are
cumulative. The two readings agree on block one and diverge afterwards --
an implementation built from the comment decodes the first frame of a
connection perfectly and then fails.

ntor carries the same trap the Swift port hit: the transcript is
B | B | X | Y | PROTOID | ID, with B twice, while the comment directly
above that code claims ID | B | Y | X | PROTOID. Both orderings produce a
well-formed 32-byte KEY_SEED. Only one interoperates.

HMAC via javax.crypto, HKDF via BouncyCastle's HKDFBytesGenerator, X25519
via the ladder from part 2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Paul Hitt
2026-09-06 13:13:02 -04:00
co-authored by Claude Opus 5
parent 44e06adf92
commit 430f167fc0
4 changed files with 513 additions and 0 deletions
@@ -0,0 +1,124 @@
package io.flamenet.obfs4
import org.bouncycastle.crypto.digests.SHA256Digest
import org.bouncycastle.crypto.generators.HKDFBytesGenerator
import org.bouncycastle.crypto.params.HKDFParameters
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
/**
* obfs4's ntor handshake.
*
* 🔴 **The reference's own comment contradicts the reference's own code, and
* the code is the protocol.** lyrebird's `ntorCommon` documents its input
* ordering as `ID | B | Y | X | PROTOID`, then builds
* `B | B | X | Y | PROTOID | ID` — with `B` appearing **twice**, because the
* buffer is constructed with it and then written to again. Port the comment and
* you get a handshake that computes cleanly, yields a respectable-looking
* 32-byte KEY_SEED, and cannot talk to a single real bridge.
*
* ⚠️ This is not Tor's ntor. It is obfs4's variant, and the difference is
* exactly that quirk.
*/
object Ntor {
const val NODE_ID_LENGTH = 20
const val KEY_SEED_LENGTH = 32
const val AUTH_LENGTH = 32
private val PROTO_ID = "ntor-curve25519-sha256-1".toByteArray()
private val T_MAC = PROTO_ID + ":mac".toByteArray()
private val T_KEY = PROTO_ID + ":key_extract".toByteArray()
private val T_VERIFY = PROTO_ID + ":key_verify".toByteArray()
private val M_EXPAND = PROTO_ID + ":key_expand".toByteArray()
data class Result(val keySeed: ByteArray, val auth: ByteArray) {
override fun equals(other: Any?): Boolean =
other is Result && keySeed.contentEquals(other.keySeed) && auth.contentEquals(other.auth)
override fun hashCode(): Int = keySeed.contentHashCode() * 31 + auth.contentHashCode()
}
/**
* The client half. `null` means the handshake must be abandoned.
*
* The caller must **also** compare `auth` against the server's, in constant
* time, with [authMatches]. A non-null result proves only that this side
* computed something.
*/
fun clientHandshake(
clientPrivate: ByteArray, clientPublic: ByteArray,
serverPublic: ByteArray, idPublic: ByteArray, nodeId: ByteArray,
): Result? {
// EXP(Y,x) | EXP(B,x)
val exp1 = Ladder.sharedSecret(clientPrivate, serverPublic)
val exp2 = Ladder.sharedSecret(clientPrivate, idPublic)
if (isZero(exp1) || isZero(exp2)) return null
return common(exp1 + exp2, nodeId, idPublic, clientPublic, serverPublic)
}
/** The server half. `null` means the handshake must be abandoned. */
fun serverHandshake(
clientPublic: ByteArray, serverPrivate: ByteArray, serverPublic: ByteArray,
idPrivate: ByteArray, idPublic: ByteArray, nodeId: ByteArray,
): Result? {
// EXP(X,y) | EXP(X,b)
val exp1 = Ladder.sharedSecret(serverPrivate, clientPublic)
val exp2 = Ladder.sharedSecret(idPrivate, clientPublic)
if (isZero(exp1) || isZero(exp2)) return null
return common(exp1 + exp2, nodeId, idPublic, clientPublic, serverPublic)
}
private fun common(
exponents: ByteArray, nodeId: ByteArray,
b: ByteArray, x: ByteArray, y: ByteArray,
): Result {
// 🔴 `b` twice. Not a typo faithfully reproduced by accident — it is the
// wire format.
val suffix = b + b + x + y + PROTO_ID + nodeId
val secretInput = exponents + suffix
val keySeed = hmac(T_KEY, secretInput)
val verify = hmac(T_VERIFY, secretInput)
val auth = hmac(T_MAC, verify + suffix + "Server".toByteArray())
return Result(keySeed, auth)
}
/**
* HKDF-SHA256 over KEY_SEED, for the framing layer's keys.
*
* ⚠️ The salt is `t_key` — the same string used as the HMAC key for
* KEY_SEED — and the info is `m_expand`. Swapping them produces key material
* that is perfectly random-looking and shared with nobody.
*/
fun kdf(keySeed: ByteArray, outputByteCount: Int): ByteArray {
val hkdf = HKDFBytesGenerator(SHA256Digest())
hkdf.init(HKDFParameters(keySeed, T_KEY, M_EXPAND))
return ByteArray(outputByteCount).also { hkdf.generateBytes(it, 0, it.size) }
}
/**
* 🔴 **Constant time, and it has to be.** `auth` arrives from the network
* and is compared against a locally computed value; a byte-at-a-time
* comparison that returns early hands an attacker a way to construct a
* valid-looking one a byte at a time.
*/
fun authMatches(a: ByteArray, b: ByteArray): Boolean {
if (a.size != b.size) return false
var diff = 0
for (i in a.indices) diff = diff or (a[i].toInt() xor b[i].toInt())
return diff == 0
}
/** A shared secret of all zeros means the peer sent a low-order point. */
private fun isZero(x: ByteArray): Boolean {
var acc = 0
for (b in x) acc = acc or b.toInt()
return acc == 0
}
private fun hmac(key: ByteArray, message: ByteArray): ByteArray =
Mac.getInstance("HmacSHA256").run {
init(SecretKeySpec(key, "HmacSHA256"))
doFinal(message)
}
}
@@ -0,0 +1,133 @@
package io.flamenet.obfs4
/**
* SipHash-2-4, and obfs4's length-obfuscation DRBG.
*
* ⚠️ **Hand-written despite BouncyCastle shipping a SipHash**, which is worth
* justifying because every other primitive in this module is BouncyCastle's.
* BC's `SipHash` is a `Mac`: `doFinal` **resets** the state. obfs4's DRBG
* writes eight bytes, finalises, writes eight more and finalises again —
* without ever resetting. Emulating that with a resetting MAC means re-hashing
* the entire history for every block, which is O(n²) in the number of frames on
* a connection. So the state machine is written out; it is fifty lines and
* there are vectors for it.
*/
class SipHash24(key: ByteArray) {
private var v0: Long
private var v1: Long
private var v2: Long
private var v3: Long
private var buffer = ByteArray(0)
private var totalBytes = 0
init {
require(key.size == 16) { "SipHash-2-4 takes a 16-byte key" }
val k0 = load64(key, 0)
val k1 = load64(key, 8)
v0 = k0 xor 0x736f6d6570736575uL.toLong()
v1 = k1 xor 0x646f72616e646f6duL.toLong()
v2 = k0 xor 0x6c7967656e657261uL.toLong()
v3 = k1 xor 0x7465646279746573uL.toLong()
}
private constructor(other: SipHash24) : this(ByteArray(16)) {
v0 = other.v0; v1 = other.v1; v2 = other.v2; v3 = other.v3
buffer = other.buffer.copyOf()
totalBytes = other.totalBytes
}
fun update(bytes: ByteArray) {
totalBytes += bytes.size
buffer += bytes
var offset = 0
while (buffer.size - offset >= 8) {
val m = load64(buffer, offset)
v3 = v3 xor m
round(); round()
v0 = v0 xor m
offset += 8
}
if (offset > 0) buffer = buffer.copyOfRange(offset, buffer.size)
}
/**
* The digest so far.
*
* 🔴 **Finalises a COPY.** The DRBG keeps writing after each read, so a
* `sum` that mutated the running state would produce a keystream correct
* for exactly one block — and the first frame of a connection would decode
* perfectly before everything after it failed.
*/
fun sum(): Long {
val c = SipHash24(this)
var b = (c.totalBytes.toLong() and 0xff) shl 56
for ((i, byte) in c.buffer.withIndex()) b = b or ((byte.toLong() and 0xff) shl (8 * i))
c.v3 = c.v3 xor b
c.round(); c.round()
c.v0 = c.v0 xor b
c.v2 = c.v2 xor 0xff
c.round(); c.round(); c.round(); c.round()
return c.v0 xor c.v1 xor c.v2 xor c.v3
}
/** ⚠️ Little-endian, matching the Go implementation obfs4 uses. */
fun sumBytes(): ByteArray {
val s = sum()
return ByteArray(8) { ((s ushr (8 * it)) and 0xff).toByte() }
}
private fun round() {
v0 += v1; v1 = java.lang.Long.rotateLeft(v1, 13); v1 = v1 xor v0
v0 = java.lang.Long.rotateLeft(v0, 32)
v2 += v3; v3 = java.lang.Long.rotateLeft(v3, 16); v3 = v3 xor v2
v0 += v3; v3 = java.lang.Long.rotateLeft(v3, 21); v3 = v3 xor v0
v2 += v1; v1 = java.lang.Long.rotateLeft(v1, 17); v1 = v1 xor v2
v2 = java.lang.Long.rotateLeft(v2, 32)
}
private fun load64(b: ByteArray, offset: Int): Long {
var v = 0L
for (i in 7 downTo 0) v = (v shl 8) or (b[offset + i].toLong() and 0xff)
return v
}
companion object {
fun hash(key: ByteArray, message: ByteArray): ByteArray =
SipHash24(key).also { it.update(message) }.sumBytes()
}
}
/**
* obfs4's length-obfuscation DRBG: SipHash-2-4 run in OFB.
*
* 🔴 **The reference's comment describes a different algorithm from its code.**
* The comment says `IV[n] = H(K, IV[n-1])` — a fresh hash of the previous block
* each time. The code never resets the hash, so block `n` is really
* `SipHash(K, IV[0] ‖ … ‖ IV[n-1])`, cumulative.
*
* The two agree on block **one** and diverge from block two. An implementation
* built from the comment decodes the first frame of a connection perfectly and
* then fails, which is about the least helpful symptom available.
*/
class LengthDrbg(seed: ByteArray) {
private val hash: SipHash24
private var ofb: ByteArray
init {
require(seed.size == 24) { "the DRBG seed is 24 bytes" }
hash = SipHash24(seed.copyOfRange(0, 16))
ofb = seed.copyOfRange(16, 24)
}
fun nextBlock(): ByteArray {
hash.update(ofb)
ofb = hash.sumBytes()
return ofb
}
/** The two bytes a frame length is masked with — the **big-endian** head. */
fun nextLengthMask(): Int {
val block = nextBlock()
return ((block[0].toInt() and 0xff) shl 8) or (block[1].toInt() and 0xff)
}
}
@@ -0,0 +1,149 @@
package io.flamenet.obfs4
import org.json.JSONObject
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertNotNull
import kotlin.test.assertNull
import kotlin.test.assertTrue
/**
* The ntor handshake, against lyrebird, from both sides.
*
* 🔴 **This is the suite that decides whether the port can talk to a real
* bridge.** Everything below it can be right while this is wrong: the maps
* round-trip, the keys are correct, and the handshake still derives a KEY_SEED
* no bridge in the world shares. obfs4 orders its transcript
* `B | B | X | Y | PROTOID | ID` — `B` twice — while the comment immediately
* above it in the reference claims `ID | B | Y | X | PROTOID`. Both orderings
* produce a well-formed 32-byte result. Only one interoperates.
*/
class NtorTest {
private fun vectors(): JSONObject {
val stream = javaClass.classLoader.getResourceAsStream("elligator-vectors.json")
assertNotNull(stream, "the vectors are missing from test resources")
return JSONObject(stream.readBytes().decodeToString())
}
private fun hex(s: String) = ByteArray(s.length / 2) {
s.substring(it * 2, it * 2 + 2).toInt(16).toByte()
}
private fun hex(b: ByteArray) = b.joinToString("") { "%02x".format(it) }
@Test
fun `the client handshake matches lyrebird`() {
val v = vectors().getJSONArray("ntor")
assertTrue(v.length() > 30, "too few handshakes to mean anything")
val mismatches = mutableListOf<String>()
for (i in 0 until v.length()) {
val o = v.getJSONObject(i)
val r = Ntor.clientHandshake(
hex(o.getString("client_private")), hex(o.getString("client_public")),
hex(o.getString("server_public")), hex(o.getString("id_public")),
hex(o.getString("node_id")),
)
if (r == null) { mismatches += "node ${o.getString("node_id")}: refused"; continue }
if (hex(r.keySeed) != o.getString("key_seed")) {
mismatches += "node ${o.getString("node_id")} KEY_SEED\n" +
" want ${o.getString("key_seed")}\n got ${hex(r.keySeed)}"
}
if (hex(r.auth) != o.getString("auth")) mismatches += "AUTH mismatch"
}
assertTrue(mismatches.isEmpty(),
"${mismatches.size} of ${v.length()} client handshakes disagreed:\n" +
mismatches.take(2).joinToString("\n"))
}
@Test
fun `the server handshake matches lyrebird`() {
val v = vectors().getJSONArray("ntor")
var mismatches = 0
for (i in 0 until v.length()) {
val o = v.getJSONObject(i)
val r = Ntor.serverHandshake(
hex(o.getString("client_public")),
hex(o.getString("server_private")), hex(o.getString("server_public")),
hex(o.getString("id_private")), hex(o.getString("id_public")),
hex(o.getString("node_id")),
)
if (r == null || hex(r.keySeed) != o.getString("key_seed") ||
hex(r.auth) != o.getString("auth")
) mismatches++
}
assertEquals(0, mismatches, "$mismatches server handshakes disagreed with lyrebird")
}
/**
* ⚠️ Both sides agreeing with each other is **not** the same as either being
* right — two identically-wrong implementations agree perfectly. This runs
* alongside the reference comparisons, never instead of them.
*/
@Test
fun `the two sides agree with each other`() {
val v = vectors().getJSONArray("ntor")
for (i in 0 until v.length()) {
val o = v.getJSONObject(i)
val c = Ntor.clientHandshake(
hex(o.getString("client_private")), hex(o.getString("client_public")),
hex(o.getString("server_public")), hex(o.getString("id_public")),
hex(o.getString("node_id")))
val s = Ntor.serverHandshake(
hex(o.getString("client_public")),
hex(o.getString("server_private")), hex(o.getString("server_public")),
hex(o.getString("id_private")), hex(o.getString("id_public")),
hex(o.getString("node_id")))
assertNotNull(c); assertEquals(c, s, "the two sides disagreed")
}
}
@Test
fun `the kdf matches lyrebird`() {
val v = vectors().getJSONArray("ntor")
var mismatches = 0
for (i in 0 until v.length()) {
val o = v.getJSONObject(i)
if (hex(Ntor.kdf(hex(o.getString("key_seed")), 72)) != o.getString("okm")) mismatches++
}
assertEquals(0, mismatches, "$mismatches KDF outputs disagreed with lyrebird")
}
/**
* 🔴 A peer that sends a low-order point drives every shared secret to zero.
* Keying from that is keying from a value the attacker chose.
*/
@Test
fun `low order points are rejected`() {
val o = vectors().getJSONArray("ntor").getJSONObject(0)
val identity = ByteArray(32)
assertNull(Ntor.clientHandshake(
hex(o.getString("client_private")), hex(o.getString("client_public")),
identity, hex(o.getString("id_public")), hex(o.getString("node_id"))),
"a zero server key must be refused")
assertNull(Ntor.serverHandshake(
identity, hex(o.getString("server_private")), hex(o.getString("server_public")),
hex(o.getString("id_private")), hex(o.getString("id_public")),
hex(o.getString("node_id"))),
"a zero client key must be refused")
}
/**
* ⚠️ `clientPublic` and `nodeId` do not affect the shared secrets at all —
* they enter only through the transcript. If the suffix were built wrongly
* they could vanish without any other test noticing.
*/
@Test
fun `every input is bound into the transcript`() {
val o = vectors().getJSONArray("ntor").getJSONObject(0)
fun run(pub: ByteArray, id: ByteArray) = Ntor.clientHandshake(
hex(o.getString("client_private")), pub,
hex(o.getString("server_public")), hex(o.getString("id_public")), id)
val base = run(hex(o.getString("client_public")), hex(o.getString("node_id")))
val otherPub = hex(o.getString("client_public")).also { it[0] = (it[0].toInt() xor 1).toByte() }
val otherId = hex(o.getString("node_id")).also { it[0] = (it[0].toInt() xor 1).toByte() }
assertFalse(base == run(otherPub, hex(o.getString("node_id"))), "X is not bound in")
assertFalse(base == run(hex(o.getString("client_public")), otherId), "node ID is not bound in")
}
}
@@ -0,0 +1,107 @@
package io.flamenet.obfs4
import org.json.JSONObject
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertNotNull
import kotlin.test.assertTrue
/**
* SipHash-2-4 and the DRBG, against the same vectors the Swift port uses.
*
* 🔴 **Hand-written crypto without vectors is indefensible.** A subtly wrong
* hash produces well-distributed output that passes every statistical glance
* and matches nobody.
*/
class SipHashTest {
private fun vectors(): JSONObject {
val stream = javaClass.classLoader.getResourceAsStream("framing-vectors.json")
assertNotNull(stream, "the framing vectors are missing from test resources")
return JSONObject(stream.readBytes().decodeToString())
}
private fun hex(s: String) = ByteArray(s.length / 2) {
s.substring(it * 2, it * 2 + 2).toInt(16).toByte()
}
private fun hex(b: ByteArray) = b.joinToString("") { "%02x".format(it) }
/**
* ⚠️ The vector lengths straddle the 8-byte block boundary and the final
* partial block on purpose — that is where a padding mistake hides, and a
* suite of round-numbered lengths would miss it entirely.
*/
@Test
fun `siphash matches the reference`() {
val v = vectors().getJSONArray("siphash")
assertTrue(v.length() > 8, "too few lengths covered")
val mismatches = mutableListOf<String>()
for (i in 0 until v.length()) {
val o = v.getJSONObject(i)
val got = SipHash24.hash(hex(o.getString("key")), hex(o.getString("input")))
if (hex(got) != o.getString("sum")) {
mismatches += "len ${o.getString("input").length / 2}: " +
"want ${o.getString("sum")} got ${hex(got)}"
}
}
assertTrue(mismatches.isEmpty(),
"${mismatches.size} of ${v.length()} results wrong:\n" +
mismatches.take(4).joinToString("\n"))
}
@Test
fun `feeding a message in pieces equals feeding it whole`() {
val v = vectors().getJSONArray("siphash")
for (i in 0 until v.length()) {
val o = v.getJSONObject(i)
val msg = hex(o.getString("input"))
if (msg.isEmpty()) continue
val h = SipHash24(hex(o.getString("key")))
for (b in msg) h.update(byteArrayOf(b))
assertEquals(o.getString("sum"), hex(h.sumBytes()),
"byte-at-a-time disagreed for a ${msg.size}-byte input")
}
}
/**
* 🔴 **The test that catches the reference's own documentation.** obfs4's
* comment says each block is a fresh `H(K, previous)`; the code never resets
* the hash, so blocks are cumulative. The two agree on block one and diverge
* afterwards, so ten blocks are checked, not one.
*/
@Test
fun `the drbg is cumulative not reset`() {
val v = vectors().getJSONArray("drbg")
assertTrue(v.getJSONObject(0).getJSONArray("blocks").length() > 5,
"one block would not distinguish the two readings")
val mismatches = mutableListOf<String>()
for (i in 0 until v.length()) {
val o = v.getJSONObject(i)
val drbg = LengthDrbg(hex(o.getString("seed")))
val blocks = o.getJSONArray("blocks")
for (j in 0 until blocks.length()) {
val got = hex(drbg.nextBlock())
if (got != blocks.getString(j)) {
mismatches += "seed ${o.getString("seed").take(16)}… block $j: " +
"want ${blocks.getString(j)} got $got"
break
}
}
}
assertTrue(mismatches.isEmpty(),
"${mismatches.size} DRBG streams diverged. A failure at block 0 means " +
"SipHash is wrong; at block 1 the hash is being reset between blocks.\n" +
mismatches.take(3).joinToString("\n"))
}
@Test
fun `the length mask is the big-endian head of the block`() {
val o = vectors().getJSONArray("drbg").getJSONObject(0)
val a = LengthDrbg(hex(o.getString("seed")))
val b = LengthDrbg(hex(o.getString("seed")))
val block = a.nextBlock()
assertEquals(((block[0].toInt() and 0xff) shl 8) or (block[1].toInt() and 0xff),
b.nextLengthMask())
}
}