Plan a scaling strategy for the relay (and revisit long-polling) #14

New Issue

2026-09-01T22:46:48Z

paulhitt commented

2026-09-01 22:46:48 +00:00

(Migrated from gitlab.com)

Ask: a plan for scaling the relay horizontally. Noted preference: away from
polling.

What it does today (read 2026-09-01, `RoutesWake.swift`)

The relay is long-poll with a Redis wake-up, and the design notes there already
weighed the alternatives:

Interval DB poll — what it falls back to with no Redis. One query per waiter per
second.
Postgres LISTEN — instant, but pins a backend connection per waiter. At
max_connections = 100 that is a ceiling of roughly 86 people with the app open.
Redis BRPOP — what it uses. Thousands of blocked clients on one process, and the
token survives between polls rather than being lost like a NOTIFY to nobody.

🔴 Two properties that any scaling plan has to preserve, because both are deliberate:

The wake is allowed to lose data. A token only says "look now"; the cursor query is
the sole source of truth. No persistence, no AOF tuning, no backup story.
Redis is optional. FN_REDIS_URL unset falls back to the interval. It is a
dependency for latency, not for function — a self-hoster must not need two services
to run a messenger.

What actually constrains scale

Held connections, not requests per second. Long-poll means concurrent open sockets is
the number that matters, and it grows with users-with-the-app-open rather than with
traffic. Worth measuring before choosing anything: current concurrent holds, memory per
held connection, and where Postgres pool exhaustion sits.

Postgres is the likely first wall, not the Swift process — max_connections is 100
and the relay shares it with the ack path.

Directions worth costing

Horizontal relays behind the existing Traefik, sharing Redis and Postgres. Wake
tokens are per-device in Redis, so any instance can serve any waiter — this may already
work without protocol change, which would make it the cheapest option by a distance.
Needs verifying, not assuming.
Replace long-poll with WebSocket or SSE. Fewer request cycles and no re-poll
latency, but it trades one held connection for another — it addresses the polling
objection without necessarily addressing the scaling one, and it is a protocol change
affecting three client implementations.
Lean harder on push. Already the path for a backgrounded app. Cannot replace the
foreground channel: web has no APNs, self-hosters have no Apple key, and §6.14's
guarantees are built on the cursor query.
PgBouncer in front of Postgres, which is cheap and probably worth doing regardless
of which direction is chosen.

Suggested first step

Measure before designing. Concurrent held connections at peak, memory per connection, and
the Postgres pool high-water mark. The current ceiling is unknown, and "we do not like
polling" and "we are running out of capacity" may not have the same answer.

⚠️ Filed during the 2026-09-01 code freeze. Planning only — no implementation.

**Ask:** a plan for scaling the relay horizontally. Noted preference: away from polling. ## What it does today (read 2026-09-01, `RoutesWake.swift`) The relay is **long-poll with a Redis wake-up**, and the design notes there already weighed the alternatives: - **Interval DB poll** — what it falls back to with no Redis. One query per waiter per second. - **Postgres `LISTEN`** — instant, but pins a backend connection per waiter. At `max_connections = 100` that is a ceiling of roughly **86 people with the app open**. - **Redis `BRPOP`** — what it uses. Thousands of blocked clients on one process, and the token survives between polls rather than being lost like a `NOTIFY` to nobody. 🔴 Two properties that any scaling plan has to preserve, because both are deliberate: 1. **The wake is allowed to lose data.** A token only says "look now"; the cursor query is the sole source of truth. No persistence, no AOF tuning, no backup story. 2. **Redis is optional.** `FN_REDIS_URL` unset falls back to the interval. It is a dependency for *latency*, not for *function* — a self-hoster must not need two services to run a messenger. ## What actually constrains scale **Held connections, not requests per second.** Long-poll means concurrent open sockets is the number that matters, and it grows with users-with-the-app-open rather than with traffic. Worth measuring before choosing anything: current concurrent holds, memory per held connection, and where Postgres pool exhaustion sits. **Postgres is the likely first wall**, not the Swift process — `max_connections` is 100 and the relay shares it with the ack path. ## Directions worth costing - **Horizontal relays behind the existing Traefik**, sharing Redis and Postgres. Wake tokens are per-device in Redis, so any instance can serve any waiter — this may already work without protocol change, which would make it the cheapest option by a distance. Needs verifying, not assuming. - **Replace long-poll with WebSocket or SSE.** Fewer request cycles and no re-poll latency, but it trades one held connection for another — it addresses the *polling* objection without necessarily addressing the *scaling* one, and it is a protocol change affecting three client implementations. - **Lean harder on push.** Already the path for a backgrounded app. Cannot replace the foreground channel: web has no APNs, self-hosters have no Apple key, and §6.14's guarantees are built on the cursor query. - **PgBouncer** in front of Postgres, which is cheap and probably worth doing regardless of which direction is chosen. ## Suggested first step Measure before designing. Concurrent held connections at peak, memory per connection, and the Postgres pool high-water mark. The current ceiling is unknown, and "we do not like polling" and "we are running out of capacity" may not have the same answer. ⚠️ Filed during the 2026-09-01 code freeze. Planning only — no implementation.

paulhitt commented

2026-09-01 23:11:44 +00:00

(Migrated from gitlab.com)

mentioned in issue flamenet-messenger-ios#11

paulhitt commented

2026-09-01 23:11:46 +00:00

(Migrated from gitlab.com)

mentioned in issue #21

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: extant2000/flamenet-relay#14

Plan a scaling strategy for the relay (and revisit long-polling) #14

What it does today (read 2026-09-01, RoutesWake.swift)

What actually constrains scale

Directions worth costing

Suggested first step

What it does today (read 2026-09-01, `RoutesWake.swift`)