EscapistsReintroduced/docs/runtime-testing.md

# Runtime testing

**Current layout:** all owned modules now live in `addons/core/addon.gproj`.
Use [single-mod setup and commands](single-mod.md) for current builds. Both
Workbench fixture selectors (`core` and `chat`) load that same project and require
the external dependencies. Five-addon and vanilla-only build instructions/results
below describe the historical layout.


## Keep the desktop unobstructed

**Desktop placement override (2026-09-20):** the user requested temporarily
disabling the right-monitor enforcement. `erri-workbench-right-monitor` is removed
from KWin's active rule list; its definition is retained, and the configuration was
backed up to `~/.config/kwinrulesrc.erri-before-disable-20260920T133915`. No placement
guard was loaded when checked. Do not re-enable this rule or load
`tools/desktop/kwin-right-monitor.js` unless the user asks to restore it. The
right-monitor instructions below describe the previous setup. Automated native
tests should continue using their private headless display.

Use terminal/file checks by default. Do not repeatedly launch Workbench or game windows on the main monitor. A visible editor should use the user's selected spare output. The user authorized the right-hand display (HDMI-A-1, 1920×1080 on this machine). Keep primary DP-3 free of test windows. A KDE rule matching `steam_app_1874910` places both editor and in-editor game windows on screen 2 before mapping and disables fullscreen. The previous configuration was backed up to `~/.config/kwinrulesrc.erri-backup`. The rule is named `erri-workbench-right-monitor` and can be removed independently in KDE Window Rules.

A later test found that Workbench can restore geometry after placement and still extend onto the main display. Automated local tests therefore now use a dedicated headless Gamescope display and a cloned disposable Wine prefix. The graphical application remains native Workbench; this is not a dedicated-server test. No performance comparison may use these headless results as equivalent to an ordinary client workload.

The additional session guard is `tools/desktop/kwin-right-monitor.js`; it minimizes matching test windows if HDMI-A-1 is unavailable. The original title-only guard missed the separate game window, so use the application-class match and initial-placement rule together. Do not claim placement is verified merely because the editor window moved. Verify every new game/dialog window. The API is documented by [KDE](https://develop.kde.org/docs/plasma/kwin/api/).

Existing missions and the original workspace are read-only references. Never save their open editor documents during successor validation. All authored successor content and fixes belong in this repository.

## Native setup

Use matching stable Arma Reforger and Arma Reforger Tools installations. Keep original Escapists and its extensions unloaded. Register `addons` as an addon search directory, and open `addons/core/addon.gproj`. Open `Worlds/ERR_Everon.ent` in the World Editor. `ERR_ValidateWorldPlugin` logs the authored locations' terrain heights; `-errPlay 1` allows that diagnostic plugin to enter play mode when deliberately invoked.

The command-line conventions follow the [official startup parameter documentation](https://community.bistudio.com/wiki/Arma_Reforger:Startup_Parameters): Workbench uses `-gproj`, `-addonsDir`, `-load` and `-plugin`. Workbench's `-load` is distinct from the game's `-world`. Launching a visible editor is an explicit test step, not part of `tools/check.py`.

## Acceptance sessions

Use the [player route playtest guide](player-route.md) for the intended normal-input sequence. It is derived from source and is not a record of a completed manual run. The current combined-addon development handoff has artifact-bound ordinary startup evidence; see [the retained playtest](playtest-build.md). Historical Core-only and five-addon results remain separate. Startup evidence does not resolve packaging/resource failures or qualify later source changes. See [current status](current-status.md) and the [packaging control](native-packaging-control.md).

Record the game/Tools versions, repository tree hash, addon list, player count, AI/vehicle counts, settings, map positions and complete logs for every run. Start from an isolated profile.

1. Join with 1, 2 and 12 players; verify one controlled prisoner each. Join late during every phase. Reconnect with the same identity both within and after native reconnect reservation expiry. Voluntary departure must not create a free extra life. Test reused transient player IDs.
2. Test conscious, unconscious, revived and dead players; forbidden respawn requests; spectators; nobody connected; all survivors dead; partial extraction; defeat; restart. Voluntary reconnect reservation and spectator following are implemented; actual remote-client verification remains required.
3. Pick the prison lock, leave the enclosure, scavenge physical folders, reveal/identify the radio, hold its area for the full continuous defence period, locate the helicopter and cross the escape boundary aboard it. Test every interrupted stage and a destroyed extraction vehicle. Check that moving or collecting objective objects cannot move the objective.
4. Observe a player, interrupt a radio hop, remove/capture a relay, let a contact expire, lose a support group and deplete support ammunition. Verify orders reach only existing forces, are not duplicated and do not use undelivered knowledge. Current native snapshots pass refreshed full-salvo firing (55 assertions), stale-report expiry (48), finite courier delivery, and an isolated vertical-flight mechanism (81); [current status](current-status.md) links their exact evidence. Operational HQ aircraft routes, broader loss cases and manual/multiplayer support acceptance remain pending.
5. Repeat restarts and long empty/occupied server periods. Compare entity, AI group, callback, waypoint, inventory and journal counts. Check JIP marker state and shared-map ownership/visibility.

## Performance

Use equivalent player/AI/vehicle workloads, settings, routes and hardware for old and successor runs. Include warm-up, repeat runs and collect frame times/server FPS with populations held constant. Record median and lower-percentile FPS plus AI simulation load and entity counts. Reducing AI population is not evidence of a faster equivalent system. No FPS improvement is claimed before those measurements exist.

Do not publish while an acceptance entry is pending or failed. Source-policy regression passes are not multiplayer or performance evidence.

## Opt-in scripted integration smoke

In local Workbench only, `-errSmoke 1` enables `ERR_NativeSmoke` alongside the validation plugin and `-errPlay 1`. Use a disposable profile: the test changes its intel and defence settings, makes the test player invulnerable, takes a scene screenshot and teleports the player and helicopter. It never runs in a release game build or dedicated server. It exercises real spawn, prison action, guaranteed physical cache documents, inventory-driven discovery, radio range, continuous occupancy, interrupted defence and physical helicopter seating. Automatic round restart reruns it.

Run `python3 tools/native_report.py <console.log> --require-completions 2` to summarize assertions and retain script errors as failures. Stop the isolated test process after the required cycles. Do not interpret completion as manual piloting, multiplayer or performance acceptance.

`tools/reference_guard.py snapshot` and `verify` can compare the original workspace's source/resource contents around a validation session. Keep the baseline outside the original workspace.

Use `tools/native_command.py workbench --exe <Workbench> --profile <disposable-profile> --smoke victory` to print the test command, or `--smoke lost-helicopter` for extraction-vehicle loss. `--smoke player-death` kills the real prisoner, checks one-life identity rules and camera activation, then waits for all-dead defeat and restart. The helicopter-loss case deletes the empty test helicopter during extraction and expects native mission defeat. `tools/archive_native.py <session-directory> <new-evidence-name>` retains raw logs, screenshots and current source hashes without overwriting earlier evidence. Stop the test before archiving and archive before changing the tested addon sources. Its nonzero exit preserves native failures.

## Windowless local runner on Linux

Clone a working Tools compatdata directory into a disposable path outside Steam, preserving symlinks (`cp -a --reflink=auto`). Never point the runner at the user's live Wine prefix. With Gamescope and Proton Experimental installed:

```sh
python3 tools/headless_workbench.py --compat /tmp/erri-headless-compat --profile /tmp/erri-workbench-profile --smoke radio-network --rounds 2
```

The runner uses Gamescope's headless backend, retains full GPU rendering on a private display, and terminates only its new process group. It rejects a concurrent editor using the same test profile, stops after failure/timeout/completion, and returns a nonzero status for any recorded native errors. `radio-network` exercises real radio equipment and interrupted delivery before the normal death/spectator test.

The user requested muted test audio. The runner creates/reuses only `erri_tests_silent`, a muted null output, sets a separate PulseAudio application ID, and verifies/reroutes/mutes test-owned streams during execution. It does not change the system default sink or other application streams.

## Core isolation

Pass `--project core --smoke victory` to `headless_workbench.py` to compile only Core and vanilla and run the separate base fixture. The full integration fixture still defaults to `--project chat`. Core now owns document recovery and objective markers; optional Intel distribution and the shared field map must not be required to advance the mission. Inspect the **Loaded addons** log section, not **Available addons**, when checking actual isolation.

## Deliberate startup failure

Use `--project core --smoke startup-failure --rounds 1` only for the negative-path fixture. It changes the test's in-memory faction catalog to an invalid combination, verifies that services stop and no players/sites/stock spawn, checks the administrator-visible reason and rejects further progression. The normal startup ERROR is expected in this test and is preserved in the native report. No source configuration or external mission is modified. This fixture does not restart, so exactly one round is required.

The Core victory fixture also holds its actual player character unconscious for 35 seconds at the radio, longer than the all-dead grace period. It checks that the physical survivor still prevents defeat, cannot advance radio progress, retains control/camera, and resumes defence after consciousness is restored without receiving another life. This uses the native controller's consciousness transition; it does not validate treatment animations, medical item consumption, bleeding or a remote revive action.

## Alternate scavenging route and visual discovery

Use `--project core --smoke cache-loss --rounds 2 --timeout 480` with the hidden runner. The fixture deletes the southern supply crate before taking any documents, then completes escape using the western cache. It verifies that neither deleted nor depleted stock is recreated, including across the usual preparation query. Both repeated rounds must independently start with fresh configured sites.

Core victory and cache-loss fixtures also inspect the three outpost barriers for upright placement and actual collision. At the medical stash they test unconsciousness, distance, a temporary physical wall, clear visibility and one discovery event. Only the fixture adds/removes that isolated wall; production discovery changes knowledge without changing scenery. The cache-loss variant uses the alternate documents, while the normal victory variant uses the original southern cache.

`--project core --smoke site-survey --rounds 1` prints a bounded terrain and obstacle survey for three areas without changing existing scenery. Candidate grid checks are placement diagnostics, not navigability or encounter-balance acceptance. Existing authored successor objects can now appear as obstacles in a repeated survey.

## Extraction capacity, results and startup rollback

Use `--project core --smoke extraction-capacity --rounds 1` for twelve physical native occupants and isolated partial-result identities. It also checks hidden native result widgets and clock settings/progression. Synthetic roster bindings do not simulate a network connection. The fixture explicitly ends its own test round; it is not evidence of natural all-dead defeat.

Use `--project core --smoke early-helicopter-loss --rounds 2` for helicopter loss during prison while a living player remains, then automatic restart. `extraction-startup-failure` substitutes an unsuitable vanilla transport in memory; `extraction-boundary-failure` uses a zero boundary radius with the real helicopter. Both startup cases require `--rounds 1`, withhold players, and verify authored population/stock/scenery rollback. Their deliberate startup errors remain in the report.

The ordinary Core victory fixture also validates the mission clock. To test nondefault startup values, first stop the private runner, snapshot all three private-profile settings files (`profile/Escapists.json`, `.bak`, `.tmp`) byte-for-byte, then set `startTimeMinutes: 375` and `timeMultiplier: 4` within its `mission` section, preserving all other sections. Complete two rounds, archive the input and results, and restore all three files exactly after the runner stops. Never modify a live or user profile. The default run separately checks that saving different next-round values does not alter the running clock. Remote clock agreement and long day/night transitions remain acceptance work.

## Radio service, task details and finite ground transport

The normal Core victory/cache-loss flow includes an isolated radio-service probe before actual tuning. It creates one additional physical character but simulates its connection membership and advances explicit service timestamps to exercise handover, unconsciousness/death, hostile affiliation, empty-server reset, live duration changes and duplicate completion. It restores the live character/configuration before the ordinary real-tick defence sequence. This is not two connected clients.

The Core fixture also opens hidden native task list/detail widgets and checks their text through prison, discovery, transmission/interruption and extraction. An existing details panel must refresh as the underlying task description changes. Local widget text is not remote replication or manual UI usability acceptance.

Use `--project core --smoke transport-loss --rounds 1` for physical UAZ driver/passenger occupancy, engine start, fuel depletion and permanent loss. The fixture then stops the site service and explicitly requests its own test reset. `--smoke transport-survey --rounds 1` temporarily omits vehicles only in its in-memory fixture definition, then surveys nearby terrain/geometry without removing scenery; it is authoring diagnostics, not a playable mission configuration.

Add `--station-teardown-diagnostics` to the hidden runner for bounded Workbench-only logging of support-station deletion validation. It forwards `-errStationTeardownDiag 1`, preserves all vanilla validation/errors, and neither repairs nor suppresses the catalog issue. [Native error investigation](native-errors.md) records the established resource mismatch and unconfirmed teardown hypothesis.

The offline [performance comparison tool](performance-comparison.md) checks paired captures and equivalent workloads before computing frame-time/FPS differences. It has only synthetic regression coverage so far; no real benchmark results have been collected. [Release qualification](release-qualification.md) now checks current native source hashes and raw logs as well as the acceptance ledger. Both tools leave current release/performance acceptance pending.

## Pending player admission

Use `--project core --smoke lifecycle --rounds 1` for the bounded physical slot, delayed finalization and cleanup probe. The fixture uses the real local controller plus explicitly owned synthetic pending actors; it does not create remote connections or grant/reset lives. See [player lifecycle](player-lifecycle.md).

## Objective panel destruction

Use `--project core --smoke objective-ui --rounds 1` for the short native close/rebind/reopen and isolated task-deletion check. It restores the actual mission task assignment and requests its own terminal test reset. See [objective widget lifecycle](objective-widget-lifecycle.md). Every new headless session captures pre-launch addon hashes and a stopped log seal; archive it before changing addon sources.


### Disposable-prefix lifecycle checks

Both hidden ordinary-game and Workbench runners take `.erri-workbench.lock` before checking that the disposable prefix is idle. Proton spells its prefix `pfx/`; only trailing separators are treated as equivalent. A busy prefix is refused before launch, never repaired automatically at startup.

After a launched test, the runners stop their own wineserver and fresh process group, then verify that no prefix process remains. For the explicitly owned `/tmp/erri-headless-compat` only, `private_prefix_cleanup.py` can reap adopted `winedevice.exe` children left behind by Proton: exact prefix, same user, installed executable and inode, fixed argv, user-systemd parent and unchanged process start identities must all match. It pins each candidate with a pidfd, rechecks the complete prefix census before each signal, and uses bounded TERM/KILL waits. Unknown or newly arriving processes stop cleanup; no other prefix or parent process is signalled. Other disposable prefixes retain ordinary shutdown and strict idle checking without this orphan-reaping path.

The runner records `cleanup_confirmed`, a structured cleanup report and any errors. Incomplete cleanup fails the run and prevents claiming a sealed stopped source/log snapshot. [The explicit historical cleanup](evidence/private-prefix-cleanup-2026-09-19-1631/review.json) removed nineteen old device helpers missed by the earlier exact-byte prefix check; historical logs are preserved. These are test-environment guarantees, not gameplay or multiplayer acceptance.


### Explicit ordinary keyboard diagnostic

After configuring the existing private prefix/profile, the loose Core/Chat runner accepts `--inspect-headless-display --observe-player-input --exercise-player-input --settle 50`. This explicitly requests only W down/up and two Tab pairs, inside the verified hidden display. The normal runner sends no input without the exercise flag. See [keyboard proof](headless-input-proof.md) for identity/focus checks, supervised key releases and exact evidence limits. Native errors remain failures. The successor storage-list adapter now passes the bounded first-open path with a rendered inventory; repeated refresh, transfers and the full route remain separate checks. A short explicit `--failure-grace 10` only retains additional diagnostic frames and does not convert a failed run into success.

## Original BMP mounted-accounting diagnostic

Use the existing hidden/muted private Workbench runner with `--project chat --smoke mounted-ammo --mounted-bmp --rounds 1 --timeout 420`, audited external addons and a fresh Prison-start profile. Perform the host-visible game/prefix preflight first and defer while any game exists. The original 180-second proof deadline is unchanged; the runner allowance includes preparation and cleanup.

This opt-in Workbench-only variant stops unstarted opening hunters and new town admissions before prison exit. It retains the initial site population, normal automatic convoy target of three, shared body cap and finite original stock. It checks all six original vehicle roots and cargo exclusion, then requires complete valid armed counts from original slot 4’s WCS BMP and that BMP’s own detached status snapshot delivered through the actual radio path. An armed UAZ cannot satisfy the BMP proof. Mine-stock, swap and courier variants are incompatible. No replacement BMP, direct report injection, ammunition refill, body refund or raised cap is used.

The source and ten focused regressions are recorded in [the preparation checkpoint](evidence/bmp-native-proof-preparation-2026-10-03/README.md). Native confirmation is pending because a host game was running at preparation time. Even a pass would qualify only this isolated accounting/radio scene, not default three-convoy admission, physical interception, multiplayer or performance.