58 lines
7.5 KiB
Markdown
58 lines
7.5 KiB
Markdown
# Release qualification audit
|
|
|
|
Run `python3 tools/release_check.py`. The command is read-only and returns **1** when any release gate is missing, malformed, failed, pending or lacks valid evidence. A passing audit returns **0**. It does not change acceptance status, create a package, publish an addon or prove the contents of a human-written test report.
|
|
|
|
The current repository is intentionally refused: all seventeen first-release acceptance cases remain pending. Clean environment/restart development evidence exists after narrowing the faction catalogs, but further production changes require a current-source full-loop qualification. Scripted development successes cannot override missing release evidence; historical native failures remain archived.
|
|
|
|
## Required ledger entries
|
|
|
|
The checker requires every existing first-release case by its stable name, from `independent_native_build` through `own_identity_release_package`. Removing a difficult case cannot qualify a release. Unknown case names are rejected rather than accepted as replacements. Valid statuses are exactly `pending`, `passed` and `failed`; missing statuses and malformed entries are errors throughout both `cases` and `development_checks`.
|
|
|
|
Every required first-release case and the `clean_native_runtime` development gate must be `passed` with a nonempty evidence list. Evidence paths must identify nonempty regular files inside this repository using canonical relative POSIX paths. Absolute paths, traversal, symlinks, directories, missing files, duplicate paths and whitespace-only evidence are refused. Duplicate JSON keys and nonstandard JSON numbers are refused too.
|
|
|
|
Other development checks may remain pending or failed, including intentionally failed historical fixtures. Historical passed checks still need safe nonempty evidence, but their source manifests are not required to match today's sources. Older failures and older passing evidence should remain archived rather than being rewritten to satisfy the gate.
|
|
|
|
## Explicit native qualification
|
|
|
|
Both `cases.independent_native_build` and `development_checks.clean_native_runtime` require two additional explicit fields when passed:
|
|
|
|
```json
|
|
{
|
|
"status": "pending",
|
|
"evidence": [
|
|
"docs/evidence/QUALIFYING-RUN/report.json",
|
|
"docs/evidence/QUALIFYING-RUN/source-sha256.json"
|
|
],
|
|
"native_report": "docs/evidence/QUALIFYING-RUN/report.json",
|
|
"source_manifest": "docs/evidence/QUALIFYING-RUN/source-sha256.json"
|
|
}
|
|
```
|
|
|
|
This is an illustrative shape, not an acceptance update; the placeholder files do not exist. Only record passed status after the corresponding acceptance work has actually passed.
|
|
|
|
The report and manifest must belong to the same evidence archive. Future native qualification also requires the fixed `source-provenance.json`, `launch-source-sha256.json`, `launch-source.json`, `runner.json.txt` and `console.log.txt` files beside them. The explicit paths must also appear in that entry's evidence list. The report must name a nonempty raw native log in the same archive. The checker independently analyzes that log with the shared native report parser and rejects missing metrics, type changes, fabricated counters or error lists that disagree with the raw log. Qualification requires every round requested by the successfully completed runner and at least one passing assertion, with no captured script/runtime, engine-severity `(E)` or `(F)` or missing end-screen errors. These are local native prerequisites; the separate multiplayer, gameplay, workload and packaging cases remain mandatory.
|
|
|
|
The source manifest must match the complete current authored addon file set and each SHA-256 hash. Changed files, added files, missing files and extra stale manifest entries all invalidate qualification. The shared `tools/source_manifest.py` helper is also used by `tools/archive_native.py`, so archival and qualification use the same inventory. Unknown resource file formats are included automatically. Generated addon-root resource databases, `.pak`/`.log` files, and `.build`, `.local`, `__pycache__`, `profiles` directories are excluded; these locations are build/runtime state and must not contain authored source resources. Source symlinks are refused.
|
|
|
|
## Launch and stopped-run provenance
|
|
|
|
The hidden runner captures the complete addon manifest **before starting the engine process**. When its new native session directory appears, it writes the already captured bytes as `launch-source-sha256.json` plus `launch-source.json` metadata. The metadata records a unique launch ID, source root, command, capture timestamp and SHA-256 of the exact manifest bytes. It never reconstructs launch source state from files encountered later.
|
|
|
|
After terminating its owned processes, the runner hashes the final addon tree and exact raw `console.log` bytes. Its `runner.json` binds the launch ID/manifest hash, process start and stop timestamps, final source digest, raw log digest and completion result. A changed final source tree makes the runner fail as `source-changed`; captured engine failures remain preserved separately.
|
|
|
|
Archive only a stopped run. `archive_native.py` validates launch metadata against runner metadata, rejects source changes between launch and stop or between launch and archival, verifies the stopped raw log digest, and copies the original launch manifest bytes into `source-sha256.json`. It also retains the launch files and writes `source-provenance.json` with `kind: "launch_snapshot"`. Qualification verifies all those relationships again, requires the report to analyze that exact sealed `console.log.txt`, and rejects a recomputed clean report attached to a different raw log.
|
|
|
|
For a session created by an older runner without launch metadata, an explicit historical import is available:
|
|
|
|
```sh
|
|
python3 tools/archive_native.py /path/to/stopped-old-session historical-name --allow-legacy-archive-time-source
|
|
```
|
|
|
|
That import writes `kind: "archive_time_only"` and `qualification: "historical_only"`; its manifest describes the files at archive time, without asserting they were loaded by the old session. Such evidence cannot satisfy future native qualification, even if its current-source hashes and smoke counters happen to match. Missing/partial/corrupt launch evidence cannot be downgraded to legacy by this flag. Already existing historical archives and their native reports remain unchanged and readable; historical development entries still do not require current launch provenance.
|
|
|
|
These checks prevent accidental stale-source and old-log mixing. They are not cryptographic authentication or continuous filesystem surveillance: edits made and reverted entirely between the start/stop snapshots cannot be detected, and a person can still alter all evidence consistently. Do not edit tested sources while the process runs; preserve collection records and review semantic acceptance. The checker makes no release-readiness claim merely from an existing screenshot, report filename or old source manifest.
|
|
|
|
## Regression checks
|
|
|
|
`python3 tests/test_release_check.py` exercises forty-six focused cases in temporary directories, including a fully qualifying synthetic ledger, omitted requirements, a failed clean-runtime gate despite all cases being passed, malformed structures/statuses, empty files, unsafe paths and symlinks, changed/added/deleted addon files, stale manifests, falsified report metrics, retained native errors, truncated logs, historical evidence, launch/stop/raw-log binding, source mutation, explicit legacy import and read-only behavior. Synthetic test fixtures are never copied into the real acceptance ledger.
|