EscapistsReintroduced/docs/evidence/tts-lag-2026-10-13/README.md

# TTS playback hitch investigation — 2026-10-13

The native benchmark reproduced a long synchronous preparation step before speech playback. Decoding a synthetic seven-second clip took **274 ms** in one callback; a thirty-second clip took **1,833 ms**. After spreading preparation across updates, the largest measured preparation call was **3 ms**, with all 36 native checks passing in a04. These are native component timings measured with `System.GetTickCount()`, not inferred from download latency or a guarantee that every game frame is hitch-free.

## Native baseline

Run [a02](a02/native-report.json) completed all three input sizes using the original production `ERR_WelcomeVoiceRules.Decode`, `FileHandle.WriteArray` and `AudioSystem.PlaySound` calls. The fixture used PCM mono, 24,000 Hz, 16-bit WAV files filled with silence. It ran locally through Workbench in a headless 1280×720/30 Hz session with a silent audio sink; it did not contact the speech service.

| Synthetic clip | Decoded WAV | Decode | Write | `PlaySound` |
|---|---:|---:|---:|---:|
| 1 second | 48,044 bytes | 41 ms | 1 ms | 0 ms |
| 7 seconds | 336,044 bytes | 274 ms | 2 ms | 1 ms |
| 30 seconds | 1,440,044 bytes | 1,833 ms | 10 ms | 0 ms |

The timer has 1 ms resolution, so a zero measurement does not prove zero cost. The source calls the complete decoder from the REST callback before the next update can play the sound. A full synthetic clip was reconstructed correctly in each case: all three native byte-count assertions passed, and the preserved native output files exactly match the decoded inputs. The baseline fixture terminates each sound immediately after starting it; this does not qualify audible speech or playback over the full utterance.

The same fixture separately measured 2,048 direct `ToAscii` reads on the full encoded string. On the thirty-second input these took 893 ms at the start and 885 ms at the end. Those diagnostic loops run **after** the decoder timing and are excluded from the table. They caution against replacing bounded-string parsing with repeated character access on the whole response.

Run [a01](a01/native-report.json) recorded only the one-second case (40 ms decode), then stopped because the legacy extraction fixture lacked a safe landing footprint for the new rescue fleet. It is retained as a failed attempt. Between a01 and a02, the **only** changed launched addon file was `Configs/ERR/Tests/LegacySites.conf`; the speech source and benchmark fixture were identical. The second run used the already-surveyed airport extraction location.

The a02 runner reports `smoke-completed`, three passing assertions and no script/runtime failures. Strict native analysis still reports 21 engine/resource errors, including missing external localization references, obsolete prefab fields, the world GUID and a Steamworks warning. This is not an engine-clean certification. Raw errors and the exact launch inputs remain in each run directory.

## Queued preparation

Run [a04](a04/native-report.json) passed the complete production-queue fixture after the ordering correction described below. It injected the same four local payloads: 1, 7 and 30 seconds, plus a second 30-second clip cancelled after a partial write. Only one clip received preparation work per update, with round-robin progress. Cancellation removed the partial file and queued work; completed audio waited for playback readiness and played exactly once. All three [preserved native WAV outputs](a04/input-output-sha256.json) exactly match their decoded source inputs by SHA-256.

| Synthetic clip | Preparation calls | Largest preparation call | Largest output chunk | `PlaySound` | Queue-start to playback |
|---|---:|---:|---:|---:|---:|
| 1 second | 24 | 3 ms | 2,736 bytes | 0 ms | 0.691 s |
| 7 seconds | 200 | 2 ms | 2,400 bytes | 0 ms | 4.799 s |
| 30 seconds | 1,256 | 3 ms | 1,584 bytes | 1 ms | 18.376 s |

Preparation now yields between small pieces; **audio can still wait before it starts**. The waits above are for several simultaneous clips sharing the queue, including the deliberately gated seven-second clip. They exclude input loading, HTTP and cloud synthesis and are not single-clip latency measurements. The batch finished in 18.385 seconds over 1,481 updates, with exactly one preparation job per update; the cancelled clip accounts for one job.

The seven-second WAV also passed the production soldier-speech spatial graph path: graph write 1 ms, initialization 2 ms, and `PlayEvent` 1 ms with a valid handle. This checks native graph compatibility, not actual soldier delivery, remote clients or audible quality.

The full game-script update still reached **110 ms**, with a **120 ms** gap between updates, because those measurements include other active mission work. The isolated preparation maximum of 3 ms therefore must not be presented as a whole-game maximum frame time. All 36 a04 assertions passed with one completion and no script/runtime failures. Its 21 strict engine/resource errors have the same normalized signatures and occurrence counts as baseline a02; they remain a separate limitation.

Run [a03](a03/native-report.json) injected four local payloads into the production preparation queue: 1, 7 and 30 seconds, plus a second 30-second clip cancelled after its first partial write. The three retained clips reached playback with a maximum measured preparation slice of 3 ms, and the seven-second clip waited behind the readiness gate. These are promising component measurements, but **a03 failed a real queue-fairness assertion and is not a passing validation run**.

The production queue used Enforce `Remove(0)`, which fills the removed slot with the last element instead of preserving order. This violated the intended round-robin schedule; the earlier host test double had modeled ordered removal. Production removal was corrected to explicit ordered operations before a04 repeated the same native fixture. Only `ERR_WelcomeVoice.c` differs between the a03 and a04 launch manifests. The a03 failure happened before final output copying and spatial playback checks, so a03 provides neither completed output hashes nor a qualified spatial result. Raw partial timing and failure evidence are retained.

## Existing client logs and separate admin-menu fix

[Filtered client events](client/client-events.txt) preserve real playback timestamps without request payloads, authentication tokens or endpoints. On October 13, download-to-play gaps were 2.061 s, 1.465 s and 1.134 s. One other request failed with HTTP 0. The client performance logs contain only cleanup memory records, with no frame-time samples. **These network-plus-preparation intervals do not establish the duration of a render hitch.**

The newest session also produced **684 admin-menu warnings** between **17:12:19.127 and 17:13:33.183**: eighteen absent `Random_` buttons requested on each of thirty-eight two-second snapshot polls. This affects every focused admin page; the TTS page merely appears in the inherited callback stack. `ReceiveSnapshot` now uses the same `ERR_Settings.Randomizable(key)` guard as menu initialization. The [exact one-line patch](client/admin-warning-fix.patch) and [existing menu-test result](client/admin-warning-validation.txt) are retained. No post-fix native UI frame-time improvement is claimed.

The [final check rerun](final-check.log.txt), after correcting ordered queue removal, retains successful focused checks followed by the existing `ERRM_AtlasLeaf.et` missing saved root ID failure. The [earlier check](full-check.log.txt) is also preserved. The repository-wide check is therefore incomplete; the unrelated asset failure is not a TTS regression. All [three changed production sources](final-production-source-matches.json) exactly match the passing a04 launch bytes.

## Evidence and reproduction

The [aggregate report](report.json) contains structured timings and run status. Each run preserves the console, script and error logs, runner result, launch metadata, external addon inputs, relevant frozen source files, benchmark fixture and exact compressed synthetic inputs/outputs. `input-output-sha256.json` records sample sizes and hashes; `artifact-sha256.json` binds archived files.

`archive.py` validates `native_provenance.read_launch` and `archive_source`, including the original source capture, stopped raw-console hash, unchanged complete addon sources, confirmed process cleanup, idle prefix and unchanged external addon inputs. It refuses to overwrite an archived run. To append another **stopped** benchmark from the corresponding `/tmp/erri-tts-bench-source-*` and `run-*` directories:

```sh
python3 docs/evidence/tts-lag-2026-10-13/archive.py a05
```

Running the archiver with no arguments revalidates preserved artifact hashes and regenerates the aggregate report. It does not launch the engine, contact a service or modify the server. The synthetic benchmark is not a multiplayer latency test, a live server deployment or a rendered-frame percentile measurement.