Attachment reads: redirect to the object store so a CDN can serve blobs directly (multi-region) #63

New Issue

2026-09-17T18:43:07Z

paulhitt commented

2026-09-17 18:43:07 +00:00

(Migrated from gitlab.com)

What

Attachment GETs are served by streaming the blob through the relay
(AttachmentStore.read returns a Response; the S3 backend fetches from the
bucket and re-serves it). So with a CDN in front the blob path is:

CDN -> relay -> object store

The relay stays in the hot path for every media download.

Why it matters (and when)

Not needed now, and not a blocker for single-region horizontal scaling, which
is already supported (FN_RELAY_REPLICAS + FN_REDIS_URL + shared Postgres,
Horizontal.swift, relay#31). This is for the k8s / multi-region step: at that
point you want media served CDN -> bucket directly, so the relay is not on
the path for every download and far regions hit a local edge instead of the
origin.

Proposed

Add a read variant that, for object-store backends, returns a redirect (302 /
signed 307) to a short-lived presigned URL, or to a public URL when the bucket
sits behind a CDN. Keep proxy-streaming as the default for the local
DirectoryAttachmentStore and as a fallback.

The AttachmentStore protocol already abstracts read, and
S3AttachmentStore already speaks SigV4, so this is presign + a redirect
response, not a new backend.
Gate it on config (e.g. FN_S3_PUBLIC_BASE for a CDN-fronted bucket, else
presign with a TTL). Local/proxy path unchanged when unset.
Keep range-request behavior working (relay#37) for whichever path serves.
Auth: the current read is authorized by the relay before streaming; a
redirect moves that trust to the signed URL's TTL, so keep the TTL short and
keep the authorization check in front of issuing the redirect.

Notes

Attachments are already sealed client-side, so the CDN/bucket only ever holds
ciphertext.
Related: relay#31 (horizontal scaling), #34 (blob backend status probe),
#37 (partial downloads).

## What Attachment GETs are served by streaming the blob **through the relay** (`AttachmentStore.read` returns a `Response`; the S3 backend fetches from the bucket and re-serves it). So with a CDN in front the blob path is: CDN -> relay -> object store The relay stays in the hot path for every media download. ## Why it matters (and when) Not needed now, and not a blocker for single-region horizontal scaling, which is already supported (`FN_RELAY_REPLICAS` + `FN_REDIS_URL` + shared Postgres, `Horizontal.swift`, relay#31). This is for the k8s / multi-region step: at that point you want media served **CDN -> bucket directly**, so the relay is not on the path for every download and far regions hit a local edge instead of the origin. ## Proposed Add a read variant that, for object-store backends, returns a redirect (302 / signed 307) to a short-lived presigned URL, or to a public URL when the bucket sits behind a CDN. Keep proxy-streaming as the default for the local `DirectoryAttachmentStore` and as a fallback. - The `AttachmentStore` protocol already abstracts read, and `S3AttachmentStore` already speaks SigV4, so this is presign + a redirect response, not a new backend. - Gate it on config (e.g. `FN_S3_PUBLIC_BASE` for a CDN-fronted bucket, else presign with a TTL). Local/proxy path unchanged when unset. - Keep range-request behavior working (relay#37) for whichever path serves. - Auth: the current read is authorized by the relay before streaming; a redirect moves that trust to the signed URL's TTL, so keep the TTL short and keep the authorization check in front of issuing the redirect. ## Notes - Attachments are already sealed client-side, so the CDN/bucket only ever holds ciphertext. - Related: relay#31 (horizontal scaling), #34 (blob backend status probe), #37 (partial downloads).

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: extant2000/flamenet-relay#63