flamenet-relay/docs/FANOUT.md

Relay-side sealed fan-out: the spike

relay#27 proposed uploading a group message once with per-device sealed keys and letting the relay copy it, and named the design question exactly: "the relay would learn the fan-out shape unless the tokens hide it, and hiding it is the entire design question."

This is the answer to that question. The tokens cannot hide it, for a structural reason rather than a cryptographic one, and the saving on offer is smaller than the issue assumes. The recommendation is to decline it as filed.

First, the premise, measured

The issue costs the problem as "a ten-person group with two devices each is twenty uploads of the same blob". Two corrections, both measured:

A second device on the same person is not a second request. /e2e/messages takes one to account and a list of that account's devices, so ten people with two devices each is ten requests, not twenty (docs/COST.md, and CostTests.swift).

The blob is not what gets uploaded N times. A picture, a voice note or a file is an attachment: uploaded once, and named from inside each recipient's sealed envelope by a token any authenticated member can fetch with. GroupFanOutTests.mediaIsUploadedOnce pins this, one 64 KB picture sent to six people leaves exactly one row in attachments, and all six download it.

So what a sender actually repeats is the envelope: a few hundred bytes of ciphertext carrying a reference. Measured at 572 bytes per member, so:

Group size	Sent today	Removable by relay-side fan-out
9 members	5,149 B	4,577 B
32 members (the cap)	~18.3 KB	~17.7 KB

Seventeen kilobytes is the entire prize, at the largest group this relay allows. Not nineteen copies of a photo.

Why the tokens cannot hide the shape

Relay-side fan-out means the relay takes one thing and puts it in N places. Doing that requires the relay to hold, at one moment, a mapping from one blob to N destinations, and that mapping is the fan-out shape. It is not a leak that sits beside the feature and might be engineered away; it is the feature, described.

Delivery tokens (§21.4) already do the part they can do. They make destinations unlinkable to accounts: the relay sees a fresh random value per message rather than a stable per-sender constant. What they cannot do is make the relay not notice that these N destinations are receiving the same bytes at the same moment, because copying is exactly the act of knowing that.

Three ways to try, and why each fails:

Split it across N requests: "deliver a copy of blob B to token T", N times. The relay now assembles the same mapping from N requests instead of one, and the sender has paid N round trips: the thing being optimised. Strictly worse than today.

Blind the blob reference per recipient. The relay has to resolve whatever each recipient presents back to the same stored bytes, so it has to know they are the same bytes. Making that untrue needs proxy re-encryption or oblivious storage, a different and far larger project than "let the relay copy it", and one with the same §21.4 prerequisite: a written transcript and review by somebody who has implemented that class of protocol before.

Delay or shuffle the copies. Obscures the timing, not the mapping. The row still says one blob, N destinations, and it says so durably.

What would be given up

Today a group message is N ordinary envelopes, each addressed to one account, each with its own ciphertext because each is separately ratcheted, and under sealed sending, with no sender the relay can see. Nothing in the database says those N rows are one message. GroupFanOutTests asserts that property directly, so anyone implementing relay#27 has to delete a passing test whose name says what is being deleted.

This matters more than 17 KB because of what was built next to it. Sealed rosters (§8.9b) exist so that a dump of this database does not reveal your groups, the claim /metadata makes in those words. Relay-side fan-out would put the group back in a row, and undo in the message path what the roster work achieved in the group path.

Being precise about what is not claimed: a running relay can already infer a lot. An identified send names the sender, and N sends from one account inside a second are an obvious group. The protection that exists is against the operator reading it later out of stored rows, and against sealed sends, where there is no sender to correlate on at all. Relay-side fan-out would break the stored-rows protection for every group message, sealed or not.

Recommendation

Decline relay#27 as filed. Seventeen kilobytes at the largest permitted group, in exchange for a durable membership record, is not a trade worth making and the tokens cannot make it a different trade.

If round trips ever become the real complaint, that is a different issue from the one filed, and it has a narrow answer worth writing down now. The cost that would bite is not the bytes but the N round trips from a phone on a high-latency mobile link. Batching several accounts into one request would fix that, and it leaks the membership list in exactly the same way, except on the identified path, where the relay already sees the sender and can correlate the burst by timing anyway.

So: if it is ever needed, allow multi-recipient batching on the identified send path only, and never on the sealed one. That buys the latency back where nothing is lost, and refuses it where something is. It should be filed against a measured complaint about round trips, not built speculatively, at 32 members the current cost is well inside what a phone does without anybody noticing.

What would change this answer

The group cap. Everything above is bounded by Group.maxMembers == 32, and GroupFanOutTests fails if that moves, so this document cannot quietly go stale behind a raised limit. A relay hosting 500-member groups is a different calculation and deserves this spike run again.

How envelopes are written today

Every envelope goes through Delivery.write (Sources/FlamenetRelay/Delivery.swift): identified and sealed sends to POST /e2e/messages, an introduction's first message (a sealed send), and federated deposits.

Inside the transaction it locks the target device rows, sorted by id, before inserting. A poll returns id > after and delete-on-ack removes id <= after, so one device's envelopes must commit in id order. Postgres takes the id at insert and shows the row at commit. Without the lock two concurrent sends to one device could commit out of order, the cursor would pass the lower id, and the next poll would delete it unread. With the lock the second writer waits for the first to commit, then takes a higher id. The cursor is still the global envelope id, so clients see no change. A group fan-out locks one device set per request, in sorted order, so two group sends cannot deadlock. SQLite has one writer and skips the lock.

After the commit it wakes held polls (in memory on one relay, through Redis with several, see RoutesWake.swift) and queues one push job on the push worker: 8 jobs run at a time and 1024 wait. A full queue drops the push and counts it; the envelope is still collected on the next poll. Shutdown flushes the queue for up to 10 seconds. Federated deposits now wake and push the same way a local send does.