All three deployment shapes, and the one that was unreachable

"Self-hosted with shared identity" could not work. Both clients decided where
accounts lived from a hardcoded list of relay hostnames, and a stranger's relay
is never on that list -- so somebody self-hosting could have the shared FlameNet
account service configured on their relay and the app would still authenticate
against the relay itself. Mode 2 existed on the server and nowhere else.

The relay already published the answer at `/.well-known/flamenet/relay`:
`accounts: shared|local` and the service URL. Nothing read it.

Now the client asks, and then checks the answer against a pinned list of account
SERVICES rather than of relays. That keeps the property the hostname list was
protecting -- an operator cannot point their users' passphrase-derived secrets
at a host they control, because a service the client does not already trust is
treated as local -- while letting any relay adopt the shared service. Unknown or
unreachable descriptor also means local, which is the safe direction: the
passphrase goes only to the server the user typed in.

relay2 is now configured as that third shape, so all three exist to test:
managed (relay.flamenet.io), self-hosted shared (relay2), self-hosted local
(relay1, relay3).

`LiveShapesTest` covers sign-up three rounds per shape, a conversation per
shape, that each relay reports what it actually is, and that a relay naming an
untrusted account service is refused. Fresh handles every round, so rounds are
repeatable by construction rather than by cleanup.

⚠️ Two things worth keeping:

Skips are announced. `return` from an unmet precondition makes a test that did
nothing report PASSED -- I had four of those reporting green before switching to
`assumeTrue`, which is the same silent-skip trap I had already criticised in the
Swift interop harness. A suite of silent skips is indistinguishable from a suite
that works.

The handle travels beside the shared-session token, never inside it. The account
service will not put a handle in a token, so a relay cannot correlate its users
with another product's; the relay keys on the product-scoped pseudonym and uses
the handle only the first time it sees one. Tested by signing in twice and
asserting the same relay account comes back.

Also deploys the discovery-document fix to relay.flamenet.io, with a database
backup and a rollback image. The transparency-log root is byte-identical after
the deploy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Paul Hitt
2026-08-24 05:28:15 -04:00
co-authored by Claude Opus 5
parent e07660559c
commit 9ced7dc3f0
6 changed files with 443 additions and 38 deletions
@@ -62,8 +62,20 @@ class RelayAccounts(
* the handle, so the relay learns an opaque id and nothing that ties the
* person to any other FlameNet product.
*/
fun sharedSession(accountToken: String): Account =
account(post("api/flamenet/v1/messenger/session", JSONObject().put("token", accountToken)))
/**
* ⚠️ The handle travels **beside** the token, not inside it. The account
* service will not put a handle in a token, so that a relay cannot
* correlate its users with another product's — the token carries a
* product-scoped pseudonym instead. The relay uses the handle only the
* first time it sees that pseudonym; after that the pseudonym decides, and
* honouring the handle would let somebody rename themselves into an
* existing account.
*/
fun sharedSession(accountToken: String, handle: String, displayName: String? = null): Account =
account(post("api/flamenet/v1/messenger/session", JSONObject()
.put("token", accountToken)
.put("handle", handle)
.also { if (!displayName.isNullOrEmpty()) it.put("display_name", displayName) }))
/**
* Ask the relay to send a reset code.
@@ -0,0 +1,90 @@
package io.flamenet.client
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.OkHttpClient
import okhttp3.Request
import org.json.JSONObject
/**
* What a relay says about itself (`GET /.well-known/flamenet/relay`).
*
* ⚠️ **A client must know where accounts live before it sends a credential
* anywhere.** The two modes stretch the passphrase differently and send the
* result to different hosts; guessing wrong means sending an authentication
* secret to a service that was never meant to receive it.
*
* This replaces a hardcoded list of hostnames. The list was safe but wrong: it
* meant a **self-hosted relay could never use the shared account service**,
* because it would never be on the list — so the "self-hosted, shared identity"
* deployment shape was unreachable in practice.
*/
object RelayDiscovery {
/**
* Account services a client will send a derived secret to.
*
* ⚠️ **This is the security boundary, and it is pinned here rather than
* taken from the relay.** The relay says *which* service it uses; this says
* which services are acceptable. Without the pin, an operator could point
* their users' passphrase-derived secrets at a host they control, and the
* whole point of stretching on-device would be lost.
*
* A self-hoster running their own account service is not blocked by this —
* their relay reports `local` and authenticates against itself.
*/
val TRUSTED_ACCOUNT_SERVICES = setOf("https://accounts.flamenet.io")
enum class Accounts { SHARED, LOCAL }
data class Descriptor(
val audience: String,
val accounts: Accounts,
val accountsUrl: String?,
val federation: String,
val protocolVersions: List<Int>,
) {
/**
* ⚠️ Shared **only** when the named service is one we already trust.
* A relay claiming `shared` with an unknown service is treated as
* local: refusing outright would break a legitimate deployment we
* simply do not know about, while believing it would hand over
* secrets.
*/
val usesSharedAccounts: Boolean
get() = accounts == Accounts.SHARED && accountsUrl in TRUSTED_ACCOUNT_SERVICES
}
/**
* Ask a relay what it is.
*
* Returns null when the relay does not answer — an older deployment, or
* something that is not a relay at all. The caller then falls back to
* local, which is the safe default: it sends the passphrase only to the
* server the user typed in.
*/
fun describe(baseUrl: String, http: OkHttpClient = RelayClient.defaultHttpClient()): Descriptor? {
val url = baseUrl.trimEnd('/').toHttpUrl().newBuilder()
.addPathSegments(".well-known/flamenet/relay").build()
return try {
http.newCall(Request.Builder().url(url).get().build()).execute().use { response ->
if (!response.isSuccessful) return null
val json = JSONObject(response.body?.string().orEmpty())
val versions = json.optJSONArray("protocolVersions")
Descriptor(
audience = json.optString("audience"),
accounts = if (json.optString("accounts") == "shared") {
Accounts.SHARED
} else {
Accounts.LOCAL
},
accountsUrl = json.optString("accountsURL").takeIf { it.isNotEmpty() },
federation = json.optString("federation", "closed"),
protocolVersions = (0 until (versions?.length() ?: 0))
.map { versions!!.getInt(it) },
)
}
} catch (_: Exception) {
null
}
}
}
@@ -0,0 +1,272 @@
package io.flamenet.client
import io.flamenet.e2e.DeviceIdentity
import io.flamenet.e2e.Vault
import javax.crypto.KeyGenerator
import javax.crypto.SecretKey
import org.junit.jupiter.api.Assumptions.assumeTrue
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
/**
* All three deployment shapes, three rounds each, repeatably.
*
* The shapes are not variations on a theme — they differ in **where a
* passphrase-derived secret is sent**, which is the one thing a messenger must
* not get wrong:
*
* - **Managed** (`relay.flamenet.io`) — accounts in the shared FlameNet
* service. One credential across Messenger and Files.
* - **Self-hosted, shared identity** (`relay2`) — somebody else's relay, still
* the shared account service. This shape was **unreachable** until the
* client started reading `/.well-known/flamenet/relay`: the old hardcoded
* hostname list could never contain a stranger's relay.
* - **Self-hosted, local** (`relay1`, `relay3`) — the relay is its own
* identity provider. The default, and not a lesser mode.
*
* ⚠️ **Every round uses fresh handles.** Reusing one makes round two fail with
* "that name is taken" — a relay answering correctly, which reads as a product
* bug. Rounds are therefore repeatable by construction rather than by cleanup.
*
* ⚠️ Managed rounds create real accounts on `accounts.flamenet.io`. Enabled
* explicitly by `FN_LIVE_MANAGED=1`, because that is not litter to leave by
* accident.
*
* FN_LIVE_FLEET=https://relay1.flamenet.io FN_LIVE_MANAGED=1 \
* ./gradlew :client:test --tests '*LiveShapesTest*'
*/
class LiveShapesTest {
/**
* ⚠️ **Skips are announced, not silent.** `return` from an unmet
* precondition makes a test that did nothing report as PASSED, and a suite
* of those is indistinguishable from a suite that works. `assumeTrue`
* reports SKIPPED, which is the truth.
*/
private fun needFleet() = assumeTrue(
System.getenv("FN_LIVE_FLEET") != null, "set FN_LIVE_FLEET to run against the fleet")
private fun needManaged() = assumeTrue(
System.getenv("FN_LIVE_MANAGED") == "1",
"set FN_LIVE_MANAGED=1 — these create real accounts on accounts.flamenet.io")
private val local = "https://relay1.flamenet.io"
private val sharedSelfHosted = "https://relay2.flamenet.io"
private val managed = "https://relay.flamenet.io"
private class MapStorage : Vault.Storage {
private val records = LinkedHashMap<String, ByteArray>()
override fun read(name: String) = records[name]
override fun write(name: String, bytes: ByteArray) { records[name] = bytes }
override fun delete(name: String) { records.remove(name) }
override fun names() = records.keys.toList()
}
private fun key(): SecretKey = KeyGenerator.getInstance("AES").apply { init(256) }.generateKey()
private fun handle(tag: String) = "shape$tag" + (System.nanoTime() % 10_000_000)
// -- What each shape claims about itself ---------------------------------
@Test
fun `each relay reports the shape it actually is`() {
needFleet()
val localDescriptor = RelayDiscovery.describe(local)
assertTrue(localDescriptor != null, "relay1 served no descriptor")
assertEquals(RelayDiscovery.Accounts.LOCAL, localDescriptor!!.accounts)
assertTrue(!localDescriptor.usesSharedAccounts)
val sharedDescriptor = RelayDiscovery.describe(sharedSelfHosted)
assertTrue(sharedDescriptor != null, "relay2 served no descriptor")
assertEquals(RelayDiscovery.Accounts.SHARED, sharedDescriptor!!.accounts)
assertTrue(sharedDescriptor.usesSharedAccounts,
"relay2 names ${sharedDescriptor.accountsUrl}, which is not trusted")
if (System.getenv("FN_LIVE_MANAGED") == "1") {
val managedDescriptor = RelayDiscovery.describe(managed)
assertTrue(managedDescriptor?.usesSharedAccounts == true,
"the managed relay does not report shared accounts")
}
}
/**
* ⚠️ A relay claiming `shared` with an account service we do not trust must
* NOT be believed. This is the check that stops an operator pointing their
* users' derived secrets at a host they control.
*/
@Test
fun `a relay naming an untrusted account service is treated as local`() {
val rogue = RelayDiscovery.Descriptor(
audience = "rogue.example",
accounts = RelayDiscovery.Accounts.SHARED,
accountsUrl = "https://accounts.rogue.example",
federation = "closed",
protocolVersions = listOf(1),
)
assertTrue(!rogue.usesSharedAccounts,
"a rogue account service was accepted — derived secrets would go there")
}
// -- Sign-up, three rounds, per shape ------------------------------------
@Test
fun `local self-hosted sign-up works three times running`() {
needFleet()
repeat(3) { round -> localRound(local, "l$round") }
}
@Test
fun `shared self-hosted sign-up works three times running`() {
needFleet(); needManaged()
repeat(3) { round -> sharedRound(sharedSelfHosted, "s$round") }
}
@Test
fun `managed sign-up works three times running`() {
needManaged()
repeat(3) { round -> sharedRound(managed, "m$round") }
}
// -- Messaging on each shape ---------------------------------------------
/**
* A conversation on every shape, so "it works" is not a claim about one
* relay's configuration.
*
* ⚠️ Deliberately three separate conversations rather than one crossing
* relays. Cross-relay delivery is federation, which is a different
* mechanism with a different failure mode; conflating them would let a
* federation problem masquerade as a messaging problem.
*/
@Test
fun `a conversation works on a local self-hosted relay`() {
needFleet()
talk(local) { tag -> localAccount(local, tag) }
}
/**
* ⚠️ Split per shape rather than one test covering all three. The shared
* shapes go through the account service, which rate-limits — correctly —
* and a single combined test lets one shape's throttle hide another
* shape's result.
*/
@Test
fun `a conversation works on a shared self-hosted relay`() {
needFleet(); needManaged()
talk(sharedSelfHosted) { tag -> sharedAccount(sharedSelfHosted, tag) }
}
@Test
fun `a conversation works on the managed relay`() {
needManaged()
talk(managed) { tag -> sharedAccount(managed, tag) }
}
private fun talk(base: String, make: (String) -> RelayAccounts.Account) {
val alice = ready(base, make("ta"))
val bob = ready(base, make("tb"))
val text = "hello on $base"
val sent = alice.second.send(bob.first, text)
assertTrue(sent.reachedSomeone, "$base delivered nothing: ${sent.failed}")
val got = bob.second.poll()
assertEquals(1, got.size, "$base: expected one message, got ${got.size}")
assertEquals(text, got.single().text)
}
/** An account plus a registered device and a ready conversation layer. */
private fun ready(base: String, account: RelayAccounts.Account):
Pair<Long, Conversations> {
val identity = DeviceIdentity.create()
val vault = Vault(MapStorage(), key())
vault.saveIdentity(identity)
val relay = RelayClient(base, account.username, account.appPassword)
relay.useRelayAuth(identity)
relay.registerDevice(identity, identity.generateOneTimePreKeys(10))
vault.saveIdentity(identity)
return account.userId to
Conversations(identity, vault, relay).also { it.myUserId = account.userId }
}
private fun localAccount(base: String, tag: String): RelayAccounts.Account {
val name = handle(tag)
return RelayAccounts(base).signUp(name, "$name@example.invalid", PASSPHRASE)
}
private fun sharedAccount(base: String, tag: String): RelayAccounts.Account {
val name = handle(tag)
val descriptor = RelayDiscovery.describe(base)!!
val accounts = AccountsClient(descriptor.accountsUrl!!)
accounts.register(name, PASSPHRASE)
return RelayAccounts(base).sharedSession(accounts.signIn(name, PASSPHRASE).token, name)
}
// -- One round of each kind ----------------------------------------------
/** Relay-local: the relay is its own identity provider. */
private fun localRound(base: String, tag: String) {
val name = handle(tag)
val account = RelayAccounts(base).signUp(name, "$name@example.invalid", PASSPHRASE)
assertEquals(name, account.username)
registerAndProve(base, account)
// Signing in again must return a working credential for the same
// account, not create a second one.
val again = RelayAccounts(base).signIn(name, PASSPHRASE)
assertEquals(account.userId, again.userId)
}
/**
* Shared identity: the passphrase is stretched on this device, only the
* derived secret reaches the account service, and the token it returns is
* exchanged with the relay for a relay credential.
*/
private fun sharedRound(base: String, tag: String) {
val name = handle(tag)
val descriptor = RelayDiscovery.describe(base)
assertTrue(descriptor?.usesSharedAccounts == true, "$base is not a shared-identity relay")
val accounts = AccountsClient(descriptor!!.accountsUrl!!)
accounts.register(name, PASSPHRASE)
val session = accounts.signIn(name, PASSPHRASE)
assertTrue(session.token.isNotEmpty(), "no session token from the account service")
val account = RelayAccounts(base).sharedSession(session.token, name)
assertTrue(account.appPassword.isNotEmpty())
registerAndProve(base, account)
// ⚠️ A second sign-in must land on the SAME relay account. The relay
// keys on the product-scoped pseudonym, not the handle; if it keyed on
// the handle, signing in twice would either fail or make a duplicate.
val second = RelayAccounts(base)
.sharedSession(accounts.signIn(name, PASSPHRASE).token, name)
assertEquals(account.userId, second.userId,
"a second sign-in produced a different relay account")
}
/** Register a device and prove the account is actually reachable. */
private fun registerAndProve(base: String, account: RelayAccounts.Account) {
val identity = DeviceIdentity.create()
val vault = Vault(MapStorage(), key())
vault.saveIdentity(identity)
val relay = RelayClient(base, account.username, account.appPassword)
relay.useRelayAuth(identity)
relay.registerDevice(identity, identity.generateOneTimePreKeys(10))
// Reachable means a stranger can fetch a bundle WITH a one-time prekey.
// A device published without them can send and can never be started
// with, which looks identical from the account's own side.
val bundles = relay.fetchKeys(account.userId)
val mine = bundles.firstOrNull { it.deviceId == identity.deviceId }
assertTrue(mine != null, "$base served no bundle for the device just registered")
assertTrue(mine!!.oneTimePreKey != null, "$base served no one-time prekey")
}
private companion object {
const val PASSPHRASE = "a-long-enough-passphrase"
}
}