EscapistsReintroduced/services/telemetry-worker/README.md

# Co10 Escape telemetry receiver

Cloudflare Worker and D1 REST API for the existing [mission telemetry protocol](../../docs/telemetry.md).
The game sends one JSON event per POST to `/api/mission-events`. Any community
server running the updated mission enrolls automatically on its first valid
event. Operators do not need an account, a manually issued token, or a profile
override. Mission → Statistics remains the administrator opt-out.

The Worker name is `co10escape-telemetry`; the bundled mod endpoint is
`https://co10escape-telemetry.zacekjakub.workers.dev/api/mission-events`.
Automatic enrollment requires a new Workshop release after 0.0.125 and the
updated Worker below. These source files alone do not change either deployment.

## Local verification

Requires Node.js 22 or newer and npm. From this directory:

```sh
npm ci
npm test
npm run check
cp .dev.vars.example .dev.vars
# Replace the example ADMIN_TOKEN with a random value (32–256 characters).
npm run migrate:local
npm run dev
```

The tests run the Worker in Miniflare/workerd with real local D1 SQL. They cover
authentication, duplicate/concurrent retries, payload conflicts, out-of-order
snapshots, final-result preservation, actual chunked HTTP requests, limits,
legacy counters, storage rollback, server revocation, credential rotation,
automatic enrollment races, ingestion rate limits, public-summary privacy and
per-map win/failure totals across servers.
They use disposable databases and synthetic identities. `npm run check` bundles
the deployable Worker without publishing it. Miniflare is pinned to the version
bundled with Wrangler so development and test runtimes match.

## Deploy

The existing account already has D1 database `erri-telemetry`, bound as `DB` in
`wrangler.jsonc`. The renamed Worker reuses this database and its history. From
this directory, with Wrangler already logged in:

```sh
npm run migrate:remote
npm run deploy
npx wrangler secret put ADMIN_TOKEN
```

The migration adds the automatic/managed registration label. Deploying the new
name creates `co10escape-telemetry`; secrets on `erri-telemetry` do not transfer.
Enter the privately saved admin credential at the prompt for the new Worker.
The old Worker is not deleted by these commands. After the new endpoint is
verified, the old Worker can be retired separately.

For a fresh Cloudflare account, run `npx wrangler login`, then
`npx wrangler d1 create co10escape-telemetry`. Update the database ID and name in
`wrangler.jsonc` and run the migration/deploy commands above. Choose unused rate
limiter namespace IDs if 1001–1003 already serve another application in the account.
An independent read-only credential is optional:

```sh
npx wrangler secret put READ_TOKEN
```

Use independently generated random secrets of 32–256 characters. Enter them at
Wrangler's prompt; do not put them in Git or in the addon. Before ADMIN_TOKEN is
set, administration and detailed reads fail closed. Public aggregate summaries
remain available. Public ingestion enrolls servers
independently of this admin credential. Wrangler prints the Worker URL, for example
`https://co10escape-telemetry.YOUR-SUBDOMAIN.workers.dev`. `/health` is a public liveness
check, not a database readiness test. The service does not enable CORS. The dashboard is served from the same Worker
and uses your entered read credential without embedding any service secret.

Deployment uses Cloudflare's documented [D1 migrations](https://developers.cloudflare.com/d1/reference/migrations/),
[Worker secrets](https://developers.cloudflare.com/workers/configuration/secrets/),
and [transactional D1 batches](https://developers.cloudflare.com/d1/worker-api/d1-database/#batch).

## Statistics dashboard

Open `https://co10escape-telemetry.zacekjakub.workers.dev/dashboard` after deploying
this Worker. Enter the existing `READ_TOKEN` (recommended) or `ADMIN_TOKEN`.
The static login page is public; every detailed data request is authenticated.
Tokens remain in tab memory, never in URLs, browser storage or bundled assets.
Refreshing the page or signing out clears the token. There are no third-party
scripts, fonts or analytics and no server-administration buttons.

- **Overview:** total servers/reports/missions, wins, losses, win rate, outcome
  breakdown and terrain records. Counts cover all stored records, not just the page.
- **Servers:** per-installation mission totals, last stored report and its event
  type, latest reported terrain/player count, registration and ingestion status.
  Search by saved name or installation ID. Unnamed servers retain their UUID.
  Updated missions automatically report the configured game-server name; automatic
  installation labels follow the newest nonblank reported name. Blank/older queued
  reports cannot erase it. Explicitly managed registration labels remain unchanged.
- **Missions:** page through history, filter by server/outcome/exact terrain,
  and open players, mods, numeric settings, coverage flags, selected snapshot JSON
  and paginated raw event history. Download the selected snapshot as JSON.
- **Players:** personal escapes and combat totals, including counter coverage.
  Coverage counts are not counts of missions with friendly-fire incidents.

Selecting a server scopes Overview, Missions and Players. The Servers tab returns
to all installations. Overview shows at most 100 terrains with an explicit notice
if truncated; all missions remain accessible through filters and pagination.
Lists show 25 records per page. Automatic refresh is disabled. Use **Refresh**
to update the current view; navigation and filters fetch their requested records.

**Last received** is the collector's timestamp of the latest newly stored event.
It includes late queued events that do not replace a completed mission snapshot;
idempotent retries do not advance it. The separately shown server-reported time
may have clock skew. Game heartbeats normally arrive every five minutes. Neither
fresh receipts nor silence prove online/offline status. A server's Enabled/Revoked
label controls ingestion and is not an online indicator. No-report servers show
“No reports yet”; absent counters and undefined win rates show a dash.

Run `npm run migrate:remote` before `npm run deploy`. Migration 0003 adds a
receipt lookup index; 0004 maintains automatic server labels from accepted events
and recovers any existing reported names without rewriting event history.
The webpage requires no mission update. Automatic name discovery requires the
updated mission release; no operator configuration is needed. Until such a server
reports, blank historical names cannot be reconstructed from its installation UUID.

## Automatic community-server enrollment

The updated addon saves a random server ID in its telemetry state and generates
an independent secret from two random UUIDs. It verifies the secret on disk in
`$profile:EscapistsTelemetryCredentials.json` before sending any request. No
receiver admin secret is included in the mod. The first valid event atomically
creates the server record and stores the event. Only the credential hash is stored
by D1. Following events and retries require that same installation credential.

Automatic records are labelled `registration: "automatic"`; operator-created
records are labelled `"managed"`. Neither label is an anti-cheat attestation.
An automatic claim cannot replace an existing server's credential or reactivate
a revoked server. Concurrent claims with different credentials yield one owner.
Preserve the profile's state and credentials together when moving a server.
Corrupted credential storage suspends delivery; it is never silently reset.

The Worker bounds request bodies and uses three Cloudflare rate limiters:
300 requests/minute per source, 10 new enrollment attempts/minute per source,
and 30 valid requests/minute per server credential. Source keys are hashes of
Cloudflare's connecting-IP header, used only in ephemeral limiter state. Limits
are approximate and local to each Cloudflare location, not a global storage
quota. Shared hosting addresses share source quotas; a 429 makes the game retry.
See [Cloudflare rate limiting](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/).

## Optional managed server credentials

Each game server has its own `serverId` in
`$profile:EscapistsTelemetryState.json`, inside `state.serverId`. Register this
existing ID; do not replace the state file, mission IDs or queued events.

The normal automatic path needs no operator action. The administrative API also
supports pre-registering an existing profile ID with a manually issued token.
For an already enrolled server, use token rotation instead of registering again.

Make an authenticated request (substitute the ID read from the profile):

```http
POST /api/servers
Authorization: Bearer ADMIN_TOKEN
Content-Type: application/json

{"serverId":"11111111-1111-4111-8111-111111111111","name":"My Escape server"}
```

The 201 response contains `serverId`, `endpoint`, and a new `token`. The raw token
is returned only on registration or rotation; D1 stores its SHA-256 hash. Save the
token directly in the server's private profile configuration:

```json
{
  "endpoint": "https://co10escape-telemetry.YOUR-SUBDOMAIN.workers.dev/api/mission-events",
  "serverName": "My Escape server",
  "token": "TOKEN_FROM_REGISTRATION"
}
```

Restrict access to that file. Restart the mission to load it. The in-game
Statistics page should report delivered events. Confirm the mission with
`GET /api/missions?serverId=SERVER_ID`. The game token is sent in the JSON envelope
as required by the existing native HTTP sender; it cannot read data, register
servers or submit events for another server ID.

`POST /api/servers/SERVER_ID/token` rotates a credential and reactivates a revoked
server. Update the private profile with the returned token and restart the
mission. `DELETE /api/servers/SERVER_ID` revokes ingestion without deleting history.
Never distribute the administrator or read credential to game servers.

## Map records and the results screen

Opening `/`, `/api/summary`, or `/api/mission-events` in a browser returns public
JSON with collected win/failure totals. No token is required for these summaries.
Use `?terrain=URL_ENCODED_ISLAND_NAME` to restrict totals to one map, and add
`&serverId=SERVER_UUID` to include that installation's record for the same map.
Without a terrain filter, the response includes overall totals and up to 100 maps
with separate win/failure counts. The response contains counts and reported island names,
not player/server names, account IDs, raw events or credentials. Full records
remain protected by admin/read credentials.

Each summary has `schemaVersion: 1`, `terrain`, `allServers` and optional `server`.
Each record contains `completed`, `wins`, `losses`, `partialMissions`, and
`winRate` (percentage; null when there are no completed rounds). The game server resolves the island name from its loaded root scene, independently
of the scenario title. Names are compared case-insensitively. Older reports
through 0.0.129 retain their world-resource identifiers; query those with the
original resource path. The receiver does not infer or rewrite historical islands.

The acknowledgement for an accepted victory/defeat event also includes a
`community` summary for that event's map, server, mission and event ID. It is
computed after storage and includes the just-finished round exactly once.
An equivalent retry returns updated totals without counting the round again.
A conflicting second terminal snapshot cannot claim inclusion. A failure to
read totals does not undo a successful event acknowledgement.

The updated mission prioritizes its final result, validates the response against
the current map/server/mission/event/outcome, and replicates a short summary to
the win/failure screen. The screen refreshes if totals arrive after it opens.
It shows map-specific wins, failures and rounded win rates for this server and
all reporting servers. Local round statistics remain available when reporting is
disabled, storage/network fails, the receiver lacks summary support, or the
mission ends before delivery completes. Community totals then show unavailable;
they are not invented as zero. This feature requires the updated Workshop build.

## API

API responses are JSON with `Cache-Control: no-store`. Administrative endpoints
require `Authorization: Bearer ADMIN_TOKEN`. Read endpoints accept ADMIN_TOKEN or
the optional READ_TOKEN. Public summaries and `/health` do not require a token;
ingestion uses each installation's own credential.

| Method and path | Purpose |
| --- | --- |
| `GET /dashboard` | Browser statistics dashboard; details require a read/admin token |
| `GET /api/dashboard` | Authenticated overview, optional `serverId`; totals, outcomes and up to 100 terrains |
| `GET /health` | Public liveness check |
| `GET /`, `GET /api/summary`, `GET /api/mission-events` | Public map win/failure summaries; optional `terrain` and `serverId` filters |
| `POST /api/mission-events` | Ingest `{ "token": "...", "event": {...} }`; first valid event enrolls an unknown server |
| `POST /api/servers` | Register `{serverId, name}` and return a new token |
| `GET /api/servers` | Read/admin access: labels, IDs, registration, mission totals, last received report and active ingestion state; optional `q` name/ID search; never token hashes |
| `POST /api/servers/:serverId/token` | Rotate token and reactivate server |
| `DELETE /api/servers/:serverId` | Revoke ingestion; retain history |
| `GET /api/missions` | Mission summaries, newest-created first |
| `GET /api/missions/:serverId/:missionId` | Authoritative selected snapshot with players, mods, settings and result |
| `GET /api/missions/:serverId/:missionId/events` | Raw event history, newest-received first |
| `GET /api/stats` | Distinct mission counts grouped by outcome, with partial-mission counts |
| `GET /api/players` | Authenticated participants' attempts, personal escapes and combat totals |

`/api/missions`, `/api/stats` and `/api/players` accept an optional `serverId` filter.
`/api/missions` also accepts an exact, case-insensitive `terrain` filter and `outcome` (`victory`, `defeat`, `preparation_failed`,
`restarted_empty`, `interrupted`, or `unfinished`). Mission, server and event lists
accept `limit` (1–100, default 50) and `before` from the preceding `nextBefore`.
Player lists sort by stable account ID and use `after` from `nextAfter`, plus
`limit`. A non-null cursor means another page may exist; the last page can be empty.
Read results use integer 0/1 for SQLite-derived flags; raw events retain booleans.

Ingestion returns 201 after a new durable event, or 200 with `duplicate: true`
after an equivalent retry. Victory/defeat acknowledgements include current
map records when available. An already used server/event key with different
content receives 409. Invalid payloads receive 400, unauthorized requests 401,
oversized bodies 413, unsupported media types 415, rate limits 429, and storage failures 503.
The current game client retries every non-2xx, including permanent errors; fix
invalid credentials/schema or inspect the queue rather than treating 4xx as
successful delivery. Request bodies are bounded while streaming, including when
there is no Content-Length. Event JSON is limited to 65,536 UTF-16 code units;
the HTTP envelope permits 264 KiB for UTF-8 expansion and the credential.

## Storage and counting rules

`events` retains validated event objects, keyed uniquely by `(serverId, eventId)`.
Envelope tokens, request headers and IP addresses are not stored by this code.
An SQL trigger updates `missions` in the same transaction. D1 must commit both
before the Worker returns 2xx. A failed projection rolls back the event too.

For each `(serverId, missionId)`, the first accepted terminal snapshot wins.
Subsequent terminal events remain in the history but cannot change its result or
count again. Before completion, snapshots advance by elapsed mission time, then
event wall-clock time; late starts/restarts cannot clear a terminal result.
Reported `previousOutcome` is retained as evidence in events, never converted
into an invented result for a missing mission. Unknown endings stay unfinished.

Player totals include authenticated, participating players from victory/defeat
snapshots only. Spectators and temporary connection IDs are excluded. A mission
victory does not imply a personal escape. Startup failures, empty restarts and
interrupted sessions do not count as gameplay wins/losses or player attempts.
An interrupted mission can still have raw combat data in its detail snapshot.

Legacy missing counters remain null, with `missionsWithKills`,
`missionsWithFriendlyKills`, `missionsWithDeaths`, `missionsWithDowned` and
`missionsWithRecoveries` showing each total's coverage. Partial and truncated
mission counts remain visible; missing players cannot be reconstructed. Results
are community reports bound to installation credentials, not independent anti-cheat attestations.

Raw events and final snapshots are retained indefinitely by default. Monitor D1
storage as usage grows. Deleting raw events also deletes their deduplication keys;
do not prune them while a server may still retry old queued events. A retention
policy should preserve deduplication tombstones if raw payloads are removed.
Keep read credentials private: mission details contain player names and account
IDs. Worker request logging is disabled in the supplied configuration. Cloudflare
platform logging/settings are managed separately in the account.

## Dashboard verification (2026-11-01)

20 local tests pass, including real Miniflare/D1 ingestion and aggregate queries,
read-only authorization, late receipt/final-result separation, pagination and
client auth/navigation/detail/error behavior using a minimal DOM boundary.
`npm run check` also exercises the exact Wrangler-bundled browser script; keep
`keep_names: false` so function serialization has no Worker-only helper references.

Migration 0003 was applied and Worker version
`8fa09cd0-f148-43d4-a6d8-685528d73618` deployed. Live dashboard HTML/JS/CSS returned
200 and matched the tested bundle byte-for-byte; health/public summary returned
200 and unauthenticated detailed routes returned 401. No production test events
were submitted. Native browser visual/accessibility verification was unavailable
in this session; the DOM tests do not establish layout quality. Authenticated
production data rendering was not exercised with the operator's private token.