Validate independent packed addons and complete physical mortar salvos
This commit is contained in:
@@ -0,0 +1,126 @@
|
||||
# Private native addon packaging
|
||||
|
||||
`tools/native_package.py` attempts the installed Workbench ResourceManager's real XPacker operation. It never publishes, changes addon identity, copies vanilla assets into the build input, or treats a development source ZIP as a PAK. A successful run produces **unqualified native package candidates**; resource contents, packed-only loading, gameplay and release acceptance remain separate.
|
||||
|
||||
## First bounded attempt
|
||||
|
||||
```sh
|
||||
python3 tools/native_package.py --project core \
|
||||
--session /tmp/erri-native-package-core-NEW \
|
||||
--compat /tmp/YOUR-IDLE-DISPOSABLE-TOOLS-PREFIX \
|
||||
--timeout 300
|
||||
```
|
||||
|
||||
Choose a new session name and an existing private cloned Tools prefix. The runner launches only when explicitly invoked; `--help` does not launch anything. `--project ai`, `intel` or `map` first packs Core, then the selected companion. `--project chat` builds Core, AI, Intel, Map, then Chat, one native invocation at a time. It stops at the first failed dependency. Timeout is 30–1800 seconds **per addon**, plus bounded owned-process/audio cleanup. It is not the overall five-addon limit.
|
||||
|
||||
The session must be a nonexistent, normalized direct child of `/tmp`, with a simple ASCII filename. Traversal aliases and redirected prefix/lock paths are refused. Validated authored payload is collected by the existing source-bundle validator, then copied into `session/source/addons`. Workbench receives that copy's `.gproj`; it never receives the original or live successor project as its build target. Existing outputs, symlinks and unreviewed addon payload formats are refused. Generated native resource databases/cache files may appear in the private copy, but changes to captured authored files fail qualification of the attempt.
|
||||
|
||||
The runner reuses the ordinary game's process guards/cleanup and the exact `.erri-workbench.lock` shared with other native runners. It refuses an active game or busy/unverifiable prefix, uses a headless Gamescope display, `-noSound`, a fresh muted sink, and routes/mutes only its own application or exclusively owned prefix streams. It never changes default audio devices, Steam's prefix, another mission, an existing editor or another process group. Backend communication and the crash reporter are disabled. It does not pass publishing, mission-play or original-mod flags.
|
||||
|
||||
## Evidence and result
|
||||
|
||||
The session retains source hashes, command/tool hashes, separate per-addon profiles and native logs, cache directories, package file hashes and `package-report.json`. Per-addon `evidence/NAME/authored-after.json` records copied authored bytes after owned cleanup; `repository-after.json` independently records the input repository after the sequence. Raw logs are hashed only after the native process has stopped. These are provenance snapshots, not tamper-proof history or proof against a modify-and-revert between observations.
|
||||
|
||||
Exit status 0 requires every selected invocation to exit naturally with code 0 (the direct Proton/Wine child, plus Gamescope if it has already exited), its Wine prefix to become idle before cleanup, a nonempty native console log, no native error/fatal/script/VM/packing failure, no changed, removed or added authored source, successful owned cleanup and nonempty `.pak` files. A small private Python child recorder writes the exact Proton argument vector and exit status independently of Gamescope; missing, malformed or mismatched records fail. Installed Proton returns its Wine subprocess status through `sys.exit(rc)`, as inspected in `Session.run_proc`/`Session.run` and the `run` branch. Once the exact native child record exists and the Wine prefix is naturally idle, a lingering owned Gamescope display may be terminated as display cleanup. This is reported as `native-child-exited` and `display_cleanup_after_native_exit=true`, not a natural display exit. The three private Wine/Steam prefix overrides are applied only inside the recorder to its Proton child, so Gamescope itself cannot falsely keep the prefix-idle guard busy. A launcher exiting while Wine children remain cannot qualify, even if cleanup subsequently kills them. Raw `error.log`/`crash.log` content also fails. No errors are filtered to manufacture success.
|
||||
|
||||
Even exit 0 means only `native-pack-candidate`. `release_qualified`, `package_contents_verified`, `native_consumption_verified` and `tool_completion_semantics_verified` remain **false**. The first native run still needs inspection of its actual completion message, package metadata/resources/platform layout and a subsequent load using packed successor addons plus vanilla only. Unknown completion behavior that never exits reaches the deadline and fails; do not promote partial output. PAK presence/hash is not a structural package parser. No run has been claimed from mocked tests.
|
||||
|
||||
## Offline tests
|
||||
|
||||
```sh
|
||||
python3 tests/test_native_package.py
|
||||
```
|
||||
|
||||
Twenty-four focused tests exercise validated copy boundaries, mutation detection, dependency order, no-overwrite and prefix locking, exact native flags, private stream targeting, error/timeout/orphaned-prefix rejection, output hashes and deliberately unqualified results. Process, audio and PAK output doubles are explicitly synthetic; these tests launch no native process and demonstrate no real package build. Add `test_native_package.py` to `tools/check.py`'s explicit list when integrating.
|
||||
|
||||
## Installed interface audit
|
||||
|
||||
Read-only, 2026-09-19. No Workbench/game process, display, network/publish operation or existing addon edit was performed.
|
||||
|
||||
## Supported native entry point
|
||||
|
||||
Stable installed Tools app **1874910**, Steam build **24870687**, contains `Workbench/ArmaReforgerWorkbenchSteamDiag.exe`. Its bundled official `docs/EnfusionScriptAPI/html/Page_CLI.html`, ResourceManager section (lines435–479), documents:
|
||||
|
||||
- `-wbModule=ResourceManager`
|
||||
- `-packAddon`: XPacker packages all data of the **current addon**.
|
||||
- `-packAddonDir DIR`: output directory for PAKs.
|
||||
- `-packAddonCacheDir DIR`: separate compressed/built-data cache.
|
||||
|
||||
All three packing flags are explicitly public. Read-only strings from the installed executable also contain these exact flag names and `-packAddon CLI detected` / `-packAddonDir CLI detected`. There is no need to invent an XPacker executable or script packing method. Only Workbench and CrashReporter executables were found in this stable Tools tree.
|
||||
|
||||
**Publishing is separate.** The same official section lists `-publishAddon` and its own output/cache/version/change-note options. None belongs in the private build command. Packaging does not grant release acceptance or publish a Workshop item.
|
||||
|
||||
`-builddata PLATFORM TARGET [ADDON]` builds runtime resources into a directory; supported documented platforms include PC, PC_WB and HEADLESS. It is not documented as a PAK pack command. Do not mislabel successful builddata/source ZIP output as native packaging.
|
||||
|
||||
## Exact project/dependency selection
|
||||
|
||||
Use `-gproj` with the selected **copied** addon's existing addon.gproj. Do not change its ID/GUID or manufacture Workshop identity. Existing project graph:
|
||||
|
||||
| Addon | ID | GUID | Dependencies (in addition to vanilla) |
|
||||
|---|---|---|---|
|
||||
| core | ERR_Core | 2528466DE220D91C | none |
|
||||
| ai | ERR_Ai | 5192138CB9D5B911 | Core |
|
||||
| intel | ERR_Intel | D31039C7C53E348A | Core |
|
||||
| map | ERR_Map | 9AAEE9041D79809E | Core |
|
||||
| chat | ERR_Chat | FA804F411BDDF2C6 | Core, AI, Intel, Map |
|
||||
|
||||
Every project also depends on vanilla `58D0FB3206B6F859`. The installed Workbench core project is `core`, GUID `5614BBCCBB55ED1C`; it is engine tooling, not a successor dependency to copy into packages. All successor .meta GUIDs/dependencies are already validated by tools/check.py and tools/source_bundle.py. Supply `-addonsDir` pointing only at the private copied successor addon root, and the same isolated profile arrangement as the existing native runner for read-only vanilla dependency discovery. Run one selected addon per invocation in order Core, AI/Intel/Map, Chat. This avoids assuming packAddon bundles dependencies.
|
||||
|
||||
## Safe private draft build command
|
||||
|
||||
Create a fresh exclusive session `/tmp/erri-native-package-<unique>/` containing `source/addons/{core,ai,intel,map,chat}`, `profile`, `cache/<addon>`, `packages/<addon>`, and evidence. Populate source from the validated source-bundle schema, including authored source/meta only, not resourceDatabase caches/profiles/evidence. Workbench may generate caches/import data beside a project, so this copy—not STAGE, actual successor or original mods—must be its working project. The source copy should be hashed before launch and inspected afterward; generated outputs must stay in this session.
|
||||
|
||||
For Core, the supported Workbench argument vector is:
|
||||
|
||||
```text
|
||||
ArmaReforgerWorkbenchSteamDiag.exe
|
||||
-gproj Z:/tmp/erri-native-package-<unique>/source/addons/core/addon.gproj
|
||||
-addonsDir Z:/tmp/erri-native-package-<unique>/source/addons
|
||||
-profile Z:/tmp/erri-native-package-<unique>/profile
|
||||
-logconsole
|
||||
-noBackend
|
||||
-disableCrashReporter
|
||||
-wbModule=ResourceManager
|
||||
-packAddon
|
||||
-packAddonDir Z:/tmp/erri-native-package-<unique>/packages/core
|
||||
-packAddonCacheDir Z:/tmp/erri-native-package-<unique>/cache/core
|
||||
```
|
||||
|
||||
Wrap the vector in the existing `gamescope --backend headless ... -- <Proton> run ...` mechanism, using a private cloned compat prefix, the existing prefix lock and private muted audio sink. No WorldEditor, mission load, play plugin or smoke flag is needed. The native runner already proves that this display isolation avoids the user's monitors; it does **not** yet prove this packaging command completed.
|
||||
|
||||
The official help says `-wbSilent` initializes modules and exits without windows; it does not explicitly promise combination/order with the native pack operation. Do not rely on that combination to qualify output before observing it. The ResourceManager `-builddata` example omits `-run`, whereas general module help says `-run` opens the selected module. Start from the packing vector above inside the headless display and retain all raw output. Whether this operation exits itself, needs `-run`, or exposes a distinct completion marker requires one bounded native experiment; document any adjustment rather than assuming CLI behavior. `-exitAfter` is documented for a particular flow-map plugin, not as a universal Workbench packing option.
|
||||
|
||||
The initial runner must require a clean native exit and nonempty actual .pak output, inspect package contents/resources/addon metadata, record hashes, and reject timeout/errors/partial output. An exit code alone or presence of an empty directory cannot establish success. Never terminate unrelated processes or overwrite a prior output. Follow-up ordinary/dedicated loading from **only packed successor outputs plus vanilla** remains separate package-consumption validation. No claim is made yet about exact generated platform directories, packer's default platform set, package metadata format, compression reproducibility, or platform resource compilation.
|
||||
|
||||
## Installed script APIs, and their limits
|
||||
|
||||
Extracted read-only from Tools `Workbench/addons/core/data.pak` into `/tmp/erri-workbench-reference`:
|
||||
|
||||
- `scripts/GameLib/generated/WorkbenchAPI/Modules/ResourceManager.c`: native `RegisterResourceFile(string absFilePath, bool bBuildRuntimeResource)`, `RebuildResourceFile(string filePath, string configuration, bool selectFiles)`, `RebuildResourceFiles(notnull array<string> filePaths, string configuration)`, `WaitForFile(string filePath, int maxTimeMs=1000)`. None is a whole-addon PAK API.
|
||||
- `scripts/GameLib/generated/WorkbenchAPI/BuildResourceContext.c`: only `event bool IsWorking()`; this class is not a packer.
|
||||
- `scripts/WorkbenchCommon/AddonBuildInfoTool.c`: actual `RunCommandline` reads `-addonPath` and `-ciInfoDst`, resolves only loaded addon roots using `GameProject.GetLoadedAddons`/GetAddonID and `$ID:` paths, and writes world build tags as JSON before Workbench.Exit(0). It does not build/pack. Its UI-only Run branch (under AddonBuildInfoTool_IN_UI) writes to every loaded addon root and should not be invoked. The CLI destination can remain in private evidence.
|
||||
|
||||
The existing source_bundle is explicitly a development source handoff, not this native build. Native packaging can be attempted privately without a publishing credential, but whether this installed packer needs backend metadata for a never-published project is still untested; retain the backend-disabled failure if it demands that rather than contacting a service or changing addon identity.
|
||||
|
||||
## Primary-source hashes (SHA-256)
|
||||
|
||||
- `/home/jakub/.local/share/Steam/steamapps/common/Arma Reforger Tools/Workbench/docs/EnfusionScriptAPI/html/Page_CLI.html`: `b1fd25c1d02e3ac687bb310496f700acaf451f66d26c572afa9ac5fd343822d1`
|
||||
- `/tmp/erri-workbench-reference/scripts/GameLib/generated/WorkbenchAPI/Modules/ResourceManager.c`: `c5d4174f2e8dc93b0664bdf0923cd676ab53a3e2cd831f2a9bc89effe0343dfe`
|
||||
- `/tmp/erri-workbench-reference/scripts/GameLib/generated/WorkbenchAPI/BuildResourceContext.c`: `1a42c10153c80df75efd9581d8a63315bf042eadafce92b130c6a34666238516`
|
||||
- `/tmp/erri-workbench-reference/scripts/WorkbenchCommon/AddonBuildInfoTool.c`: `332aa13076bedabf0fcb2d8747552284546d2f26737b6b6707f21e43e13e6e23`
|
||||
|
||||
Self-review before first execution also verifies the command separator indexes refer to Proton and the Workbench executable before inserting the recorder wrapper. The captured authored subset of source_bundle.build exactly matches source_bundle.inventory (bundle README/manifest are excluded); both native-copy changes and changes to the input repository fail the attempt. The same validated inventory is reapplied to the copy so generated cache exclusions cannot hide newly added scripts/resources.
|
||||
|
||||
## Explicit silent-Workbench experiment
|
||||
|
||||
`--silent-workbench` adds only the officially documented `-wbSilent`; it is off by default. Use a fresh session, for example the same first-attempt command with `--silent-workbench` appended. The installed help promises initialization and exit without windows, but does not establish compatibility/order with `-packAddon`. This option is a bounded native experiment: successful packing markers, actual PAK output, exact native child exit and every existing error/source/cleanup gate still need to pass. It never filters GUI/resource errors.
|
||||
|
||||
The first nonsilent Core attempt (session `/tmp/erri-native-package-core-20260919-1046`) reported packaging success and produced a structurally inspectable PAK, then emitted 24 native UI resource-leak errors. Its original runner waited for the outer display despite the native child record already reporting zero. The follow-up runner handles that completion distinction and the optional silent initialization, but neither change retroactively qualifies the earlier run. See [pak-inventory.md](pak-inventory.md) for the separate read-only structural inspection.
|
||||
|
||||
## Diagnostic continuation across addons
|
||||
|
||||
`--continue-on-failure` is explicit and defaults to false. It allows a requested dependency closure to produce diagnostic outputs even when an earlier addon has native validation errors, such as the shutdown leaks reproduced by the [empty-addon control](native-packaging-control.md). For example, add `--project chat --continue-on-failure` to a fresh private-session invocation to attempt Core, AI, Intel, Map and Chat in that order, using the same validated private source copy. Native commands still load source dependencies from that private copy; this does not validate consumption of an earlier failed PAK.
|
||||
|
||||
Continuation requires safely observed native/process completion, confirmed owned cleanup, no runner error, an unchanged copied authored inventory, and a fresh idle-prefix check before the next addon. An unconfirmed cleanup, runner exception, busy/unverifiable prefix, changed source, timeout or interruption stops the sequence regardless of the option. It never restarts an addon, reuses another session's outputs, filters errors, or relaxes `candidate_ok`.
|
||||
|
||||
Every completed attempt retains its full report and raw errors. The session report records `continue_on_failure`, `stopped_reason` and `skipped_projects`. Any failed attempt keeps `all_candidates_created=false` and the overall exit status nonzero even when later addons produce candidates; `release_qualified` remains false in every case. Without the option, the first failed candidate still stops the dependency sequence. Offline regressions exercise full ordered continuation with mixed results, default stop, unsafe cleanup/runner/source/completion stop, a prefix becoming busy between addons, and unchanged qualification rules. These mocked tests do not constitute native companion packaging evidence.
|
||||
Reference in New Issue
Block a user