56 lines
7.6 KiB
Markdown
56 lines
7.6 KiB
Markdown
# Runtime testing
|
||
|
||
## Keep the desktop unobstructed
|
||
|
||
Use terminal/file checks by default. Do not repeatedly launch Workbench or game windows on the main monitor. A visible editor should use the user's selected spare output. The user authorized the right-hand display (HDMI-A-1, 1920×1080 on this machine). Keep primary DP-3 free of test windows. A KDE rule matching `steam_app_1874910` places both editor and in-editor game windows on screen 2 before mapping and disables fullscreen. The previous configuration was backed up to `~/.config/kwinrulesrc.erri-backup`. The rule is named `erri-workbench-right-monitor` and can be removed independently in KDE Window Rules.
|
||
|
||
A later test found that Workbench can restore geometry after placement and still extend onto the main display. Automated local tests therefore now use a dedicated headless Gamescope display and a cloned disposable Wine prefix. The graphical application remains native Workbench; this is not a dedicated-server test. No performance comparison may use these headless results as equivalent to an ordinary client workload.
|
||
|
||
The additional session guard is `tools/desktop/kwin-right-monitor.js`; it minimizes matching test windows if HDMI-A-1 is unavailable. The original title-only guard missed the separate game window, so use the application-class match and initial-placement rule together. Do not claim placement is verified merely because the editor window moved. Verify every new game/dialog window. The API is documented by [KDE](https://develop.kde.org/docs/plasma/kwin/api/).
|
||
|
||
Existing missions and the original workspace are read-only references. Never save their open editor documents during successor validation. All authored successor content and fixes belong in this repository.
|
||
|
||
## Native setup
|
||
|
||
Use matching stable Arma Reforger and Arma Reforger Tools installations. Keep original Escapists and its extensions unloaded. Register `addons` as an addon search directory, and open `addons/chat/addon.gproj`. Open `Worlds/ERR_Everon.ent` in the World Editor. `ERR_ValidateWorldPlugin` logs the authored locations' terrain heights; `-errPlay 1` allows that diagnostic plugin to enter play mode when deliberately invoked.
|
||
|
||
The command-line conventions follow the [official startup parameter documentation](https://community.bistudio.com/wiki/Arma_Reforger:Startup_Parameters): Workbench uses `-gproj`, `-addonsDir`, `-load` and `-plugin`. Workbench's `-load` is distinct from the game's `-world`. Launching a visible editor is an explicit test step, not part of `tools/check.py`.
|
||
|
||
## Acceptance sessions
|
||
|
||
Record the game/Tools versions, repository tree hash, addon list, player count, AI/vehicle counts, settings, map positions and complete logs for every run. Start from an isolated profile.
|
||
|
||
1. Join with 1, 2 and 12 players; verify one controlled prisoner each. Join late during every phase. Reconnect with the same identity both within and after native reconnect reservation expiry. Voluntary departure must not create a free extra life. Test reused transient player IDs.
|
||
2. Test conscious, unconscious, revived and dead players; forbidden respawn requests; spectators; nobody connected; all survivors dead; partial extraction; defeat; restart. Voluntary reconnect reservation and spectator following are implemented; actual remote-client verification remains required.
|
||
3. Pick the prison lock, leave the enclosure, scavenge physical folders, reveal/identify the radio, hold its area for the full continuous defence period, locate the helicopter and cross the escape boundary aboard it. Test every interrupted stage and a destroyed extraction vehicle. Check that moving or collecting objective objects cannot move the objective.
|
||
4. Observe a player, interrupt a radio hop, remove/capture a relay, let a contact expire, lose a support group and deplete support ammunition. Verify orders reach only existing forces, are not duplicated and do not use undelivered knowledge. Aircraft/mortar/logistics cases remain blocked until their controllers are migrated.
|
||
5. Repeat restarts and long empty/occupied server periods. Compare entity, AI group, callback, waypoint, inventory and journal counts. Check JIP marker state and shared-map ownership/visibility.
|
||
|
||
## Performance
|
||
|
||
Use equivalent player/AI/vehicle workloads, settings, routes and hardware for old and successor runs. Include warm-up, repeat runs and collect frame times/server FPS with populations held constant. Record median and lower-percentile FPS plus AI simulation load and entity counts. Reducing AI population is not evidence of a faster equivalent system. No FPS improvement is claimed before those measurements exist.
|
||
|
||
Do not publish while an acceptance entry is pending or failed. Source-policy regression passes are not multiplayer or performance evidence.
|
||
|
||
## Opt-in scripted integration smoke
|
||
|
||
In local Workbench only, `-errSmoke 1` enables `ERR_NativeSmoke` alongside the validation plugin and `-errPlay 1`. Use a disposable profile: the test changes its intel and defence settings, makes the test player invulnerable, takes a scene screenshot and teleports the player and helicopter. It never runs in a release game build or dedicated server. It exercises real spawn, prison action, guaranteed physical cache documents, inventory-driven discovery, radio range, continuous occupancy, interrupted defence and physical helicopter seating. Automatic round restart reruns it.
|
||
|
||
Run `python3 tools/native_report.py <console.log> --require-completions 2` to summarize assertions and retain script errors as failures. Stop the isolated test process after the required cycles. Do not interpret completion as manual piloting, multiplayer or performance acceptance.
|
||
|
||
`tools/reference_guard.py snapshot` and `verify` can compare the original workspace's source/resource contents around a validation session. Keep the baseline outside the original workspace.
|
||
|
||
Use `tools/native_command.py workbench --exe <Workbench> --profile <disposable-profile> --smoke victory` to print the test command, or `--smoke lost-helicopter` for extraction-vehicle loss. `--smoke player-death` kills the real prisoner, checks one-life identity rules and camera activation, then waits for all-dead defeat and restart. The helicopter-loss case deletes the empty test helicopter during extraction and expects native mission defeat. `tools/archive_native.py <session-directory> <new-evidence-name>` retains raw logs, screenshots and current source hashes without overwriting earlier evidence. Stop the test before archiving and archive before changing the tested addon sources. Its nonzero exit preserves native failures.
|
||
|
||
## Windowless local runner on Linux
|
||
|
||
Clone a working Tools compatdata directory into a disposable path outside Steam, preserving symlinks (`cp -a --reflink=auto`). Never point the runner at the user's live Wine prefix. With Gamescope and Proton Experimental installed:
|
||
|
||
```sh
|
||
python3 tools/headless_workbench.py --compat /tmp/erri-headless-compat --profile /tmp/erri-workbench-profile --smoke radio-network --rounds 2
|
||
```
|
||
|
||
The runner uses Gamescope's headless backend, retains full GPU rendering on a private display, and terminates only its new process group. It rejects a concurrent editor using the same test profile, stops after failure/timeout/completion, and returns a nonzero status for any recorded native errors. `radio-network` exercises real radio equipment and interrupted delivery before the normal death/spectator test.
|
||
|
||
The user requested muted test audio. The runner creates/reuses only `erri_tests_silent`, a muted null output, sets a separate PulseAudio application ID, and verifies/reroutes/mutes test-owned streams during execution. It does not change the system default sink or other application streams.
|