12 KiB
Runtime testing
Keep the desktop unobstructed
Use terminal/file checks by default. Do not repeatedly launch Workbench or game windows on the main monitor. A visible editor should use the user's selected spare output. The user authorized the right-hand display (HDMI-A-1, 1920×1080 on this machine). Keep primary DP-3 free of test windows. A KDE rule matching steam_app_1874910 places both editor and in-editor game windows on screen 2 before mapping and disables fullscreen. The previous configuration was backed up to ~/.config/kwinrulesrc.erri-backup. The rule is named erri-workbench-right-monitor and can be removed independently in KDE Window Rules.
A later test found that Workbench can restore geometry after placement and still extend onto the main display. Automated local tests therefore now use a dedicated headless Gamescope display and a cloned disposable Wine prefix. The graphical application remains native Workbench; this is not a dedicated-server test. No performance comparison may use these headless results as equivalent to an ordinary client workload.
The additional session guard is tools/desktop/kwin-right-monitor.js; it minimizes matching test windows if HDMI-A-1 is unavailable. The original title-only guard missed the separate game window, so use the application-class match and initial-placement rule together. Do not claim placement is verified merely because the editor window moved. Verify every new game/dialog window. The API is documented by KDE.
Existing missions and the original workspace are read-only references. Never save their open editor documents during successor validation. All authored successor content and fixes belong in this repository.
Native setup
Use matching stable Arma Reforger and Arma Reforger Tools installations. Keep original Escapists and its extensions unloaded. Register addons as an addon search directory, and open addons/chat/addon.gproj. Open Worlds/ERR_Everon.ent in the World Editor. ERR_ValidateWorldPlugin logs the authored locations' terrain heights; -errPlay 1 allows that diagnostic plugin to enter play mode when deliberately invoked.
The command-line conventions follow the official startup parameter documentation: Workbench uses -gproj, -addonsDir, -load and -plugin. Workbench's -load is distinct from the game's -world. Launching a visible editor is an explicit test step, not part of tools/check.py.
Acceptance sessions
Record the game/Tools versions, repository tree hash, addon list, player count, AI/vehicle counts, settings, map positions and complete logs for every run. Start from an isolated profile.
- Join with 1, 2 and 12 players; verify one controlled prisoner each. Join late during every phase. Reconnect with the same identity both within and after native reconnect reservation expiry. Voluntary departure must not create a free extra life. Test reused transient player IDs.
- Test conscious, unconscious, revived and dead players; forbidden respawn requests; spectators; nobody connected; all survivors dead; partial extraction; defeat; restart. Voluntary reconnect reservation and spectator following are implemented; actual remote-client verification remains required.
- Pick the prison lock, leave the enclosure, scavenge physical folders, reveal/identify the radio, hold its area for the full continuous defence period, locate the helicopter and cross the escape boundary aboard it. Test every interrupted stage and a destroyed extraction vehicle. Check that moving or collecting objective objects cannot move the objective.
- Observe a player, interrupt a radio hop, remove/capture a relay, let a contact expire, lose a support group and deplete support ammunition. Verify orders reach only existing forces, are not duplicated and do not use undelivered knowledge. Aircraft/mortar/logistics cases remain blocked until their controllers are migrated.
- Repeat restarts and long empty/occupied server periods. Compare entity, AI group, callback, waypoint, inventory and journal counts. Check JIP marker state and shared-map ownership/visibility.
Performance
Use equivalent player/AI/vehicle workloads, settings, routes and hardware for old and successor runs. Include warm-up, repeat runs and collect frame times/server FPS with populations held constant. Record median and lower-percentile FPS plus AI simulation load and entity counts. Reducing AI population is not evidence of a faster equivalent system. No FPS improvement is claimed before those measurements exist.
Do not publish while an acceptance entry is pending or failed. Source-policy regression passes are not multiplayer or performance evidence.
Opt-in scripted integration smoke
In local Workbench only, -errSmoke 1 enables ERR_NativeSmoke alongside the validation plugin and -errPlay 1. Use a disposable profile: the test changes its intel and defence settings, makes the test player invulnerable, takes a scene screenshot and teleports the player and helicopter. It never runs in a release game build or dedicated server. It exercises real spawn, prison action, guaranteed physical cache documents, inventory-driven discovery, radio range, continuous occupancy, interrupted defence and physical helicopter seating. Automatic round restart reruns it.
Run python3 tools/native_report.py <console.log> --require-completions 2 to summarize assertions and retain script errors as failures. Stop the isolated test process after the required cycles. Do not interpret completion as manual piloting, multiplayer or performance acceptance.
tools/reference_guard.py snapshot and verify can compare the original workspace's source/resource contents around a validation session. Keep the baseline outside the original workspace.
Use tools/native_command.py workbench --exe <Workbench> --profile <disposable-profile> --smoke victory to print the test command, or --smoke lost-helicopter for extraction-vehicle loss. --smoke player-death kills the real prisoner, checks one-life identity rules and camera activation, then waits for all-dead defeat and restart. The helicopter-loss case deletes the empty test helicopter during extraction and expects native mission defeat. tools/archive_native.py <session-directory> <new-evidence-name> retains raw logs, screenshots and current source hashes without overwriting earlier evidence. Stop the test before archiving and archive before changing the tested addon sources. Its nonzero exit preserves native failures.
Windowless local runner on Linux
Clone a working Tools compatdata directory into a disposable path outside Steam, preserving symlinks (cp -a --reflink=auto). Never point the runner at the user's live Wine prefix. With Gamescope and Proton Experimental installed:
python3 tools/headless_workbench.py --compat /tmp/erri-headless-compat --profile /tmp/erri-workbench-profile --smoke radio-network --rounds 2
The runner uses Gamescope's headless backend, retains full GPU rendering on a private display, and terminates only its new process group. It rejects a concurrent editor using the same test profile, stops after failure/timeout/completion, and returns a nonzero status for any recorded native errors. radio-network exercises real radio equipment and interrupted delivery before the normal death/spectator test.
The user requested muted test audio. The runner creates/reuses only erri_tests_silent, a muted null output, sets a separate PulseAudio application ID, and verifies/reroutes/mutes test-owned streams during execution. It does not change the system default sink or other application streams.
Core isolation
Pass --project core --smoke victory to headless_workbench.py to compile only Core and vanilla and run the separate base fixture. The full integration fixture still defaults to --project chat. Core now owns document recovery and objective markers; optional Intel distribution and the shared field map must not be required to advance the mission. Inspect the Loaded addons log section, not Available addons, when checking actual isolation.
Deliberate startup failure
Use --project core --smoke startup-failure --rounds 1 only for the negative-path fixture. It changes the test's in-memory faction catalog to an invalid combination, verifies that services stop and no players/sites/stock spawn, checks the administrator-visible reason and rejects further progression. The normal startup ERROR is expected in this test and is preserved in the native report. No source configuration or external mission is modified. This fixture does not restart, so exactly one round is required.
The Core victory fixture also holds its actual player character unconscious for 35 seconds at the radio, longer than the all-dead grace period. It checks that the physical survivor still prevents defeat, cannot advance radio progress, retains control/camera, and resumes defence after consciousness is restored without receiving another life. This uses the native controller's consciousness transition; it does not validate treatment animations, medical item consumption, bleeding or a remote revive action.
Alternate scavenging route and visual discovery
Use --project core --smoke cache-loss --rounds 2 --timeout 480 with the hidden runner. The fixture deletes the southern supply crate before taking any documents, then completes escape using the western cache. It verifies that neither deleted nor depleted stock is recreated, including across the usual preparation query. Both repeated rounds must independently start with fresh configured sites.
Core victory and cache-loss fixtures also inspect the three outpost barriers for upright placement and actual collision. At the medical stash they test unconsciousness, distance, a temporary physical wall, clear visibility and one discovery event. Only the fixture adds/removes that isolated wall; production discovery changes knowledge without changing scenery. The cache-loss variant uses the alternate documents, while the normal victory variant uses the original southern cache.
--project core --smoke site-survey --rounds 1 prints a bounded terrain and obstacle survey for three areas without changing existing scenery. Candidate grid checks are placement diagnostics, not navigability or encounter-balance acceptance. Existing authored successor objects can now appear as obstacles in a repeated survey.
Extraction capacity, results and startup rollback
Use --project core --smoke extraction-capacity --rounds 1 for twelve physical native occupants and isolated partial-result identities. It also checks hidden native result widgets and clock settings/progression. Synthetic roster bindings do not simulate a network connection. The fixture explicitly ends its own test round; it is not evidence of natural all-dead defeat.
Use --project core --smoke early-helicopter-loss --rounds 2 for helicopter loss during prison while a living player remains, then automatic restart. extraction-startup-failure substitutes an unsuitable vanilla transport in memory; extraction-boundary-failure uses a zero boundary radius with the real helicopter. Both startup cases require --rounds 1, withhold players, and verify authored population/stock/scenery rollback. Their deliberate startup errors remain in the report.
The ordinary Core victory fixture also validates the mission clock. To test nondefault startup values, first stop the private runner, snapshot all three private-profile settings files (profile/rereintroduced.json, .bak, .tmp) byte-for-byte, then seed a valid primary settings file with startTimeMinutes: 375 and timeMultiplier: 4. Complete two rounds, archive the input and results, and restore all three files exactly after the runner stops. Never modify a live or user profile. The default run separately checks that saving different next-round values does not alter the running clock. Remote clock agreement and long day/night transitions remain acceptance work.