39 lines
5.3 KiB
Markdown
39 lines
5.3 KiB
Markdown
# Read-only native PAK inventory
|
|
|
|
`tools/pak_inventory.py` validates the observed Enfusion PAK container structure, bounded decompression and SHA-256 of every indexed file without extracting payloads. It does not verify that an entry is a valid prefab, compiled script, resource identity, platform binary or loadable scenario. A successful report always keeps `platform_or_resource_semantics_verified` and `release_qualified` false.
|
|
|
|
```sh
|
|
python3 tools/pak_inventory.py /tmp/PRIVATE-SESSION/packages/core/data.pak \
|
|
--output /tmp/NEW-pak-inventory.json
|
|
python3 tests/test_pak_inventory.py
|
|
```
|
|
|
|
The optional JSON report must not exist. Without it, JSON goes to stdout. Input is opened read-only, without following the final symlink; nonregular inputs such as FIFOs are refused without blocking. No extracted file or directory is created. Report hashes identify observed content, not publisher authenticity or authorship.
|
|
|
|
## Observed format supported
|
|
|
|
The format was inspected from installed stable Tools build 24870687, and the successor's first actual private Core packing candidate. This is a strict observed-format implementation, not a claim of an official universal PAK specification.
|
|
|
|
- `FORM`, a big-endian 32-bit length equal to the complete file size minus eight, then `PAC1`.
|
|
- Exactly `HEAD`, `DATA`, `FILE` chunks, in that order. Each has a four-byte tag and big-endian 32-bit payload length; no guessed alignment padding or additional chunks.
|
|
- A 28-byte `HEAD` with prefix `03 00 01 00` and four trailing zero bytes. The intervening twenty bytes are opaque and preserved as hexadecimal, not interpreted as identities, checksums or platform claims.
|
|
- `FILE` begins with two zero bytes followed by a little-endian root child count. Tree nodes have an unsigned byte kind (0 directory, 1 file), unsigned byte UTF-8 component length and that name. A directory then has a little-endian 32-bit child count. A file then has a 24-byte descriptor: little-endian offset/stored size/expanded size, six observed zero bytes, compression flag/level, and four opaque trailing bytes. Those trailing bytes are **not** assumed to be CRCs.
|
|
- Observed compression tuples `(0,0)` and `(1,6)` mean raw and zlib respectively. Other flags, levels, reserved fields or versions fail as unsupported. The parser requires exact decompressed size, native zlib checksum/end-of-stream and absence of trailing or concatenated compressed streams.
|
|
- Every DATA byte must belong to exactly one indexed, contiguous file extent. Overlap, unindexed gaps, unindexed tails and deduplicated/sparse variants are refused rather than guessed.
|
|
|
|
Paths reject traversal, separators inside components, drive syntax, NUL/control bytes, empty/dot names, trailing dot/space, Windows reserved device names, noncanonical Unicode normalization and case-insensitive collisions. Files and directories share the same collision namespace.
|
|
|
|
## Bounds
|
|
|
|
Default limits are 4 GiB archive, 32 MiB FILE metadata, 250,000 total tree nodes, depth 64, 1 GiB expanded data per file, 8 GiB aggregate expanded data and a 1024:1 expansion ratio. Metadata is validated before payload decompression. Expanded bytes are streamed in chunks of at most 1 MiB and never written to disk; malicious streams exceeding a smaller declared size are detected after at most that declared size plus one byte. The Python API exposes explicit positive `Limits` for callers needing smaller budgets. Directory depth cannot be raised above 128. The CLI deliberately uses the reviewed defaults.
|
|
|
|
The file's size and nanosecond modification/change times are checked before and after inspection. This detects ordinary concurrent modification, not an adversarial modification/reversion with forged timestamps. Unknown format variants, malformed metadata, checksum errors, size overflow/bounds violations and excessive expansion fail closed.
|
|
|
|
## Actual read-only observations
|
|
|
|
The installed Workbench `addons/core/data.pak` was read and hashed: 1,064 files, 1,192 total tree nodes, 25,423,457 expanded bytes; archive SHA-256 `a2f2aee2b1043ebee24995581fc53fb5f3713195fda432e6c2b4c5acdcc64ea4`. Its payload was not extracted or copied into this repository. Installed game `addons/data/data.pak` and Workbench `shaders-dx12.pak` were inspected for structural metadata only; those observations also used HEAD 03000100 and the same descriptor flag pairs.
|
|
|
|
The first successor Core candidate at `/tmp/erri-native-package-core-20260919-1046/packages/core/data.pak` passes this structural/content-hash inventory: 107 files, 122 total tree nodes, 605,769 expanded bytes; archive SHA-256 `a071b5a7298543734bb72c72bb87c0dbfc447b3a176870299abd0a3def4fe623`. Inventory report: `/tmp/erri-core-first-pak-inventory.json`. That build emitted 24 native UI resource-leak errors and its original runner remained waiting on the display wrapper. **This archive inventory does not clear those errors or qualify that packaging run.** Native packed-only loading is separate.
|
|
|
|
Eleven focused tests use clearly synthetic archives: raw/compressed nested content, truncation at every byte, header/chunk/version errors, unsafe/colliding paths, descriptor flags/bounds/overlaps/gaps, zlib checksum/truncation/trailing streams, large streamed output, metadata and expansion limits, underspecified decompression bombs, FIFO/symlink refusal, no-overwrite JSON behavior and concurrent input mutation. They invoke no engine and fabricate no real build measurements.
|