Remove location and other identifying data — starting with the edge access log #19

New Issue

2026-09-01T23:06:33Z

paulhitt commented

2026-09-01 23:06:33 +00:00

(Migrated from gitlab.com)

Ask: find and remove location and other identifying data. Broader than the media
work already done.

Already done (2026-09-01) — not this ticket

EXIF/GPS is stripped from every outgoing photo and video on both platforms, and it is a
tested guarantee rather than a side effect of resizing. Story videos are remuxed to drop
container metadata; a failed scrub refuses the post rather than sending the original. See
docs/BUGS-2026-09-01.md in the iOS repo.

The larger one: the edge log knows every visitor's IP

/srv/prod/traefik/docker-compose.yaml runs --accesslog=true in JSON, and every line
carries client IP plus User-Agent. An IP is location data — coarse, but it is the
identifying field most systems actually leak, and it covers every request to
relay.flamenet.io, including app traffic from people who never visit the site.

⚠️ It is there for a reason, and the reason is written down. Only User-Agent is kept —
headers default to drop, so no Referer and no cookies — because it is the only way to
tell an installed app checking android-latest.json from a browser doing the same, which
is the whole of "did anyone actually install it". Removing it blindly loses that.

The questions worth answering:

Retention. How long are these kept, and is anything rotating or deleting them? A
request log that never expires is a permanent record of who talked to the relay and
when. This is the highest-value change and probably the easiest.
IP truncation. Traefik can be fronted or post-processed to drop the last octet.
Keeps the funnel analysis, loses the individual.
Do app requests need logging at all? The install-funnel argument covers
downloads/* and the public pages. It does not obviously cover /e2e/*, which is
where actual users' traffic lives — and that is exactly the traffic whose IP nobody
should be keeping. Separate rules per router may get both.

Other identifying surfaces worth checking

geogate — sign-up is restricted to a list of countries, which implies IP
geolocation somewhere. What does it retain, and for how long?
CrowdSec — bans by IP and therefore stores IPs, with its own retention.
Push tokens (push_tokens, 3 rows) — an APNs token is a stable per-install
identifier held by Apple as well as by us. Confirm it is deleted on sign-out; the code
calls forgetPushToken, but the erasing and non-erasing sign-out doors take different
paths as of 2026-09-01.
Attachment filenames — the legacy @photo path sends a filename; check nothing
carries the original camera name.
users has 20 dead tuples against 2 live rows — old row versions persist until
vacuumed. Overlaps #17.

Suggested first move

Retention on the access log, then decide about /e2e/*. Those two give most of the
benefit and neither costs the install-funnel data the log exists for.

⚠️ Freeze in effect — investigate and propose, do not change. Traefik config changes affect
every site on the box, not just the messenger.

Related: #17 (plaintext inventory), #18 (metadata minimisation), #16 (security review).

**Ask:** find and remove location and other identifying data. Broader than the media work already done. ## Already done (2026-09-01) — not this ticket EXIF/GPS is stripped from every outgoing photo and video on both platforms, and it is a tested guarantee rather than a side effect of resizing. Story videos are remuxed to drop container metadata; a failed scrub refuses the post rather than sending the original. See `docs/BUGS-2026-09-01.md` in the iOS repo. ## The larger one: the edge log knows every visitor's IP `/srv/prod/traefik/docker-compose.yaml` runs `--accesslog=true` in JSON, and every line carries **client IP** plus **User-Agent**. An IP is location data — coarse, but it is the identifying field most systems actually leak, and it covers every request to `relay.flamenet.io`, including app traffic from people who never visit the site. ⚠️ **It is there for a reason, and the reason is written down.** Only User-Agent is kept — headers default to `drop`, so no Referer and no cookies — because it is the only way to tell an installed app checking `android-latest.json` from a browser doing the same, which is the whole of "did anyone actually install it". Removing it blindly loses that. **The questions worth answering:** - **Retention.** How long are these kept, and is anything rotating or deleting them? A request log that never expires is a permanent record of who talked to the relay and when. This is the highest-value change and probably the easiest. - **IP truncation.** Traefik can be fronted or post-processed to drop the last octet. Keeps the funnel analysis, loses the individual. - **Do app requests need logging at all?** The install-funnel argument covers `downloads/*` and the public pages. It does not obviously cover `/e2e/*`, which is where actual users' traffic lives — and that is exactly the traffic whose IP nobody should be keeping. Separate rules per router may get both. ## Other identifying surfaces worth checking - **`geogate`** — sign-up is restricted to a list of countries, which implies IP geolocation somewhere. What does it retain, and for how long? - **CrowdSec** — bans by IP and therefore stores IPs, with its own retention. - **Push tokens** (`push_tokens`, 3 rows) — an APNs token is a stable per-install identifier held by Apple as well as by us. Confirm it is deleted on sign-out; the code calls `forgetPushToken`, but the erasing and non-erasing sign-out doors take different paths as of 2026-09-01. - **Attachment filenames** — the legacy `@photo` path sends a filename; check nothing carries the original camera name. - **`users` has 20 dead tuples against 2 live rows** — old row versions persist until vacuumed. Overlaps #17. ## Suggested first move Retention on the access log, then decide about `/e2e/*`. Those two give most of the benefit and neither costs the install-funnel data the log exists for. ⚠️ Freeze in effect — investigate and propose, do not change. Traefik config changes affect every site on the box, not just the messenger. Related: #17 (plaintext inventory), #18 (metadata minimisation), #16 (security review).

paulhitt commented

2026-09-04 22:06:57 +00:00

(Migrated from gitlab.com)

Retention done and applied 2026-09-04. Written up in docs/PLAINTEXT.md (!48). Leaving open for the per-router work described at the end.

Retention — the question you called highest-value and easiest

Measured: 590 MB across 7 files, every line carrying a client IP, pruned weekly. Now pruned daily — the window is a day instead of a week.

🔴 IP truncation is a trap, and I nearly walked into it

Your suggestion was to drop or truncate the IP and keep the funnel analysis. Traefik supports exactly that, and it looks like a complete win: ClientHost goes, User-Agent stays, install-funnel intact.

CrowdSec reads this exact log. /srv/prod/crowdsec/acquis.yaml:

source: docker
container_name:
  - edge-traefik-1

Drop the field and CrowdSec parses lines with no address in them and bans nobody — with nothing anywhere reporting a problem. IP banning would be off and every dashboard would look identical.

⚠️ So the access log serves two masters with opposite needs. The funnel wants the User-Agent and not the IP; CrowdSec wants the IP and not the User-Agent. One stream, so time is the only lever that helps both — which is why retention was the change and field-dropping was not.

⚠️ Truncation was verified, not assumed. Truncating a file a tailer is reading can leave it seeking past EOF. Across the change with fresh traffic, CrowdSec's reads went 1,642,488 → 1,642,529 and parsed 1,585,311 → 1,585,351. It survived. (And cscli metrics rounds to 1.64M, which a dozen lines cannot move — the check only means anything with -o json.)

The other surfaces you listed — checked and cleared

geogate mounts only GeoLite2 and a blocked page, both read-only, and writes nothing persistent. Container log 56 KB.
CrowdSec's own alerts do name IPs, but the store is self-limiting: 50 alerts, oldest about six hours old. Nothing unbounded.

Still open: per-router rules

Your third question — do /e2e/* requests need logging at all — is the right one and is not a one-line change. That is real user traffic and exactly the IPs nobody should keep; the install-funnel argument only ever covered downloads/* and the public pages. But CrowdSec has to keep seeing /e2e/*, because that is where credential-stuffing shows up. Two sinks and a filter, not a flag.

**Retention done and applied 2026-09-04.** Written up in `docs/PLAINTEXT.md` (!48). Leaving open for the per-router work described at the end. ### Retention — the question you called highest-value and easiest Measured: **590 MB across 7 files**, every line carrying a client IP, pruned **weekly**. Now pruned **daily** — the window is a day instead of a week. ### 🔴 IP truncation is a trap, and I nearly walked into it Your suggestion was to drop or truncate the IP and keep the funnel analysis. Traefik supports exactly that, and it looks like a complete win: `ClientHost` goes, `User-Agent` stays, install-funnel intact. **CrowdSec reads this exact log.** `/srv/prod/crowdsec/acquis.yaml`: source: docker container_name: - edge-traefik-1 Drop the field and CrowdSec parses lines with no address in them and bans nobody — **with nothing anywhere reporting a problem.** IP banning would be off and every dashboard would look identical. ⚠️ So the access log serves two masters with opposite needs. The funnel wants the **User-Agent** and not the IP; CrowdSec wants the **IP** and not the User-Agent. One stream, so **time is the only lever that helps both** — which is why retention was the change and field-dropping was not. ⚠️ **Truncation was verified, not assumed.** Truncating a file a tailer is reading can leave it seeking past EOF. Across the change with fresh traffic, CrowdSec's `reads` went 1,642,488 → 1,642,529 and `parsed` 1,585,311 → 1,585,351. It survived. (And `cscli metrics` rounds to `1.64M`, which a dozen lines cannot move — the check only means anything with `-o json`.) ### The other surfaces you listed — checked and cleared - **geogate** mounts only GeoLite2 and a blocked page, both read-only, and writes nothing persistent. Container log 56 KB. - **CrowdSec's own alerts** do name IPs, but the store is self-limiting: 50 alerts, oldest about six hours old. Nothing unbounded. ### Still open: per-router rules Your third question — do `/e2e/*` requests need logging at all — is the right one and is not a one-line change. That is real user traffic and exactly the IPs nobody should keep; the install-funnel argument only ever covered `downloads/*` and the public pages. But CrowdSec has to keep seeing `/e2e/*`, because that is where credential-stuffing shows up. Two sinks and a filter, not a flag.

paulhitt commented

2026-09-07 12:47:51 +00:00

(Migrated from gitlab.com)

mentioned in merge request !75

paulhitt commented

2026-09-07 13:32:11 +00:00

(Migrated from gitlab.com)

mentioned in issue #59

paulhitt commented

2026-09-07 13:32:40 +00:00

(Migrated from gitlab.com)

Pass 1 done and written up in docs/IDENTIFYING-DATA.md (MR !75), which also fixes the biggest finding.

🔴 Push tokens outlive the device and the account. Measured on production: 8 rows in push_tokens against 4 live devices. Six point at a device that no longer exists; four of those at a user_id that does not join to users at all. Oldest nine days, nothing to collect it. An APNs token is a stable per-install identifier Apple holds as well as we do, so a row nobody can use is an identifier kept for no reason.

Cause: all three paths that remove a device — revokeDevice, deleteAccount, and the retention expiry sweep — delete prekeys and envelopes and forgot this table. !75 makes each take the token with it, and sweeps the orphans already there.

⚠️ A second cause is client-side and not fixed there. iOS signOut clears store.appPassword as step 1 — deliberately; it is what makes the keys left behind unable to fetch anything — and then calls forgetPushToken(), a try? call on an authenticated route. With a live cached relay token it works; once that expires there is nothing left to mint another, it 401s, and the try? swallows it. So it works most of the time and fails invisibly the rest, which is worse than failing always. Same shape as the attachment 401. Needs its own ticket.

✅ The edge access log now has a bound — and the timer has never deleted anything. Both halves are true and reporting only the first would be wrong. Hourly, exit 0, and "pruned 0 MB" on every run since it was installed:

Sep 07 08:00:00 prune-edge-logs[498580]: pruned 0 MB of rotated edge access log
Sep 07 09:00:41 prune-edge-logs[753684]: pruned 0 MB of rotated edge access log

The reason is benign: edge-traefik-1 was recreated 13 hours earlier for the obfs4 deploy, discarding its log history, so only …-json.log and …-json.log.1 exist and the script deliberately starts deleting at .2. ⚠️ "Exit 0" plus "0 MB" is not evidence of a working sweep, so the glob was run against a fixture rather than assumed:

before:  .log  .log.1  .log.2  .log.3  .log.9  .log.10  .log.11
after:   .log  .log.1                      (freed 10 MB)

It removes .2 upward including the two-digit rotations. It will start biting at roughly 18 hours of traffic.

Proposal, not applied. edge-traefik-1 runs max-file: 10, max-size: 100m — Docker's own cap is 1 GB, about 3½ days of client IPs at current volume. Lowering max-file to 2 puts the bound at ~18 hours in Docker itself and leaves the timer as belt-and-braces rather than the only thing standing between us and 3½ days. It costs a container recreate, and that container fronts every site on the box, so it is written down rather than done.

✅ Stale here: users no longer carries dead tuples — autovacuum has run. Only envelopes does (14 live, 14 dead).

Still open, untouched: /e2e/* logging (needs separate routers — accesslog.fields is global), CrowdSec decision retention (cscli config show returned nothing readable over ssh; left open rather than guessed), and attachment filenames on the legacy @photo path.

edge-geogate-1 logs to json-file with max-file: 2, max-size: 5m — a hard 10 MB ceiling, bounded by volume rather than time.

Pass 1 done and written up in `docs/IDENTIFYING-DATA.md` (MR !75), which also fixes the biggest finding. **🔴 Push tokens outlive the device *and* the account.** Measured on production: **8 rows in `push_tokens` against 4 live devices.** Six point at a device that no longer exists; four of those at a `user_id` that does not join to `users` at all. Oldest nine days, nothing to collect it. An APNs token is a stable per-install identifier Apple holds as well as we do, so a row nobody can use is an identifier kept for no reason. Cause: all three paths that remove a device — `revokeDevice`, `deleteAccount`, and the retention expiry sweep — delete prekeys and envelopes and forgot this table. !75 makes each take the token with it, and sweeps the orphans already there. ⚠️ **A second cause is client-side and not fixed there.** iOS `signOut` clears `store.appPassword` as step 1 — deliberately; it is what makes the keys left behind unable to fetch anything — and then calls `forgetPushToken()`, a `try?` call on an authenticated route. With a live cached relay token it works; once that expires there is nothing left to mint another, it 401s, and the `try?` swallows it. So it works most of the time and fails invisibly the rest, which is worse than failing always. Same shape as the attachment `401`. Needs its own ticket. **✅ The edge access log now has a bound — and the timer has never deleted anything.** Both halves are true and reporting only the first would be wrong. Hourly, exit 0, and **"pruned 0 MB" on every run since it was installed**: ``` Sep 07 08:00:00 prune-edge-logs[498580]: pruned 0 MB of rotated edge access log Sep 07 09:00:41 prune-edge-logs[753684]: pruned 0 MB of rotated edge access log ``` The reason is benign: `edge-traefik-1` was recreated 13 hours earlier for the obfs4 deploy, discarding its log history, so only `…-json.log` and `…-json.log.1` exist and the script deliberately starts deleting at `.2`. ⚠️ "Exit 0" plus "0 MB" is not evidence of a working sweep, so the glob was run against a fixture rather than assumed: ``` before: .log .log.1 .log.2 .log.3 .log.9 .log.10 .log.11 after: .log .log.1 (freed 10 MB) ``` It removes `.2` upward including the two-digit rotations. It will start biting at roughly 18 hours of traffic. **Proposal, not applied.** `edge-traefik-1` runs `max-file: 10, max-size: 100m` — Docker's own cap is **1 GB, about 3½ days of client IPs at current volume**. Lowering `max-file` to `2` puts the bound at ~18 hours in Docker itself and leaves the timer as belt-and-braces rather than the only thing standing between us and 3½ days. It costs a container recreate, and that container fronts every site on the box, so it is written down rather than done. **✅ Stale here:** `users` no longer carries dead tuples — autovacuum has run. Only `envelopes` does (14 live, 14 dead). **Still open, untouched:** `/e2e/*` logging (needs separate routers — `accesslog.fields` is global), CrowdSec decision retention (`cscli config show` returned nothing readable over ssh; left open rather than guessed), and attachment filenames on the legacy `@photo` path. `edge-geogate-1` logs to `json-file` with `max-file: 2, max-size: 5m` — a hard 10 MB ceiling, bounded by volume rather than time.

paulhitt commented

2026-09-07 13:33:02 +00:00

(Migrated from gitlab.com)

mentioned in issue flamenet-messenger-ios#71

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: extant2000/flamenet-relay#19