Skip to content

feat(artifacts): read UnityFS AssetBundle containers - #7699

Merged
gensokyo-dev merged 6 commits into
morluto:mainfrom
blankym:feat-unityfs-reader
Oct 28, 2026
Merged

gensokyo-dev merged 6 commits into
morluto:mainfrom
blankym:feat-unityfs-reader

Conversation

@blankym

@blankym blankym commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Problem and outcome

Standalone UnityFS AssetBundles were opaque artifacts. The existing CLI and MCP inspection/extraction workflows now enumerate and recover format 6–8 members without running Unity. Version 9 has a different block-table layout and remains unsupported.

Approach and contract

  • One parsed header/table owner preserves producer-specific alignment and encryption flags, raw node flags, nullable unknown facts, and virtual offsets. Virtual member offsets are not physical carrier locations (source_location: null).
  • Directory buffers/rows/path projections are admitted before allocation. Member reads have separate plaintext and I/O-work budgets, bounded active streams, directly streamed plain ranges and one immutable completed-block cache. Empty-output rows retain their source count/offsets but do not multiply member traversal work.
  • Streams remain owned through decoder/I/O retirement, cancellation and close; failed descriptor cleanup remains retryable. Carrier freshness is checked at member open and completion.
  • Producer-owned content availability and traversal coverage remain distinct. Unsupported UnityFS members retain their names/sizes with unavailable hashes and explicit unknown coverage; ASAR missing companions preserve the existing per-entry semantics. Generic shutdown/cleanup failures propagate.
  • Byte-based artifact detection, shared reader construction, desktop header probes and existing retained filtered/paged views use the same contracts. Caller-selected extraction outputs use the existing output ownership/cleanup workflow; source bytes are preserved.

Executed validation

  • Source, compiled CLI, full/compact stdio MCP, Godot PCK, ASAR cleanup/identity and desktop checks ran locally. Latest focused workflow run: 42/42; strengthened empty-block/I/O/lifecycle suite: 17/17. Existing Next-context acceptance cases pass 4/4 after distinguishing availability from traversal coverage; the earlier change reproduced 2 failures while the unchanged matching-main baseline passed 4/4.
  • npm run check:fast, npm run docs:check, changed-file formatting and bounded single-thread typed lint passed. Final-head CI run 40656542039 passed, including all eight portable shards and CI required; Greptile review is 5/5 with no unresolved threads. Merged as e78ac1a.
  • Public UnityPy corpus pinned at 417d59522a998fdee9a9c2b29fe35f3063222457: atlas_test, banner_1 and char_118_yuki.ab have six raw members verified by independent UnityPy 1.24.2 SHA-256 and actual CLI/MCP extraction. atlas_test source SHA-256: 61ee5afb5f5c64f0a4f65f039d09b2509198363e713423289c895067ac215177.
  • Public xinzexi_2_n_tex contains LZMA data: both names/sizes remain visible with unavailable hashes. Five independent UnityPy repacks additionally cover format 7/8, front/end tables, padding and unsupported LZMA data/table behavior. These are library-produced files, not Unity engine executions.
  • Real compact MCP retained views return about 911–1017 estimated tokens for selected facts versus 2130–2301 for complete inspection responses. The entire four-file verification session, including catalog and text/structured response duplication, was about 87,397 estimated tokens (catalog 50,939); tokenx estimates are not billing counts.
  • Final 94fab716c was compared with the original reader on the same main (2b8fb387b, baseline merge fa5296412): 12 alternating completed inspect/extract/SHA/cleanup tasks over atlas_test and char_118_yuki.ab, three fresh-process repetitions per file/lane. Before → after: elapsed 4.99–6.06 → 5.11–6.05 s; CPU 6.52–7.11 → 6.73–7.10 s; sampled process-family RSS 260.5–282.0 → 262.2–280.0 MiB. Request+stdout estimates 3253/3305 → 3237/3289 tokens. First/repeat runs do not imply flushed OS caches; ranges overlap, so no general speedup is claimed.
  • A completed final four-file MCP task measured 8.80 s elapsed, 10.86 s CPU and 452.4 MiB sampled process-family RSS, including SDK client, server, catalog, selected views, SHA checks, cleanup and measurement/reporting. CPU uses wait4 plus sampler CPU; family RSS is sampled every 10 ms. Node 24.22.1 heaps were capped at 2 GiB, test workers at two. See UnityFS bounds and scope.

Verification scope excludes real Unity engine execution, version 9, decryption, SerializedFile object/texture decoding, and embedded-carrier discovery. Input hashes, extraction cleanup receipts, per-task elapsed/CPU/family-RSS and token estimates are retained locally; no binary fixtures or planning artifacts are committed.

Author readiness

Human owner: @blankym

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

inspect-artifact and extract-artifact now recognize the UnityFS signature
and treat format versions 6 through 9 as containers. Header facts (format
version, player/engine strings, declared size, dataflags, blocks-info
bounds and the declared 16-byte data hash) are retained alongside a
directory of CAB-* and .resS members. Member bytes stream on demand:
only the blocks covering the requested member are read and decompressed
with bounded expansion and cancellation.

Bundles declaring asset-bundle encryption keep header facts and report
unknown directory coverage. LZMA blocks-info reports unknown coverage;
LZMA member reads fail as unavailable while the directory stays
readable. Malformed counts, out-of-stream members and declared sizes
beyond the file fail as format errors. UnityFS classifies by bytes in
root detection, inventory graph construction and desktop inspection,
which emits a unityfs-header fact.

Validation: tsc --noEmit, oxlint (no new findings on touched files),
17 reader tests covering LZ4HC/uncompressed tables, blocks-info at end,
format 6 without padding alignment, multi-block members, malformed
input, unsupported compression, cancellation and abandoned streams;
full artifacts/domain/conformance suites pass. Verified against one
real YooAsset bundle (Unity 2021.3.32f1, format 8, LZ4HC): the
streamed SerializedFile contained its managed assembly byte-identical
to a UnityPy extraction. Other producer variants unverified.
@blankym
blankym marked this pull request as ready for review October 28, 2026 02:33
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-28T02:33:35.059388Z
We couldn’t complete this review. Comment @codex review to try again.
0dcac0e Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@greptile-apps

greptile-apps Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The reviewed changes appear safe to merge, with both numbered findings addressed and no new blocking issue found.

Summary

Adds standalone UnityFS format 6–8 inspection and extraction through the shared CLI/MCP workflows.

  • UnityFS bundles expose readable members without running Unity.
  • UnityFS bundles enter the existing inspection and extraction workflows.
  • Unavailable member bytes stay separate from reader and cleanup failures.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[CLI or MCP input] --> B[Byte-based detection]
  B --> C[UnityFS reader]
  C --> D[Bounded header and directory parsing]
  D --> E[Named members and source facts]
  E --> F{Content supported?}
  F -->|Yes| G[Bounded member stream]
  G --> H[Hash or extract bytes]
  F -->|No| I[Unavailable hash and unknown coverage]
Loading

Reviews (3) · Last reviewed commit: "fix(artifacts): preserve coverage author..." · Reviewed by Greptile

Comment thread src/artifacts/UnityFsStructures.ts Outdated
Comment thread src/artifacts/UnityFsArtifactReader.ts Outdated
Comment thread src/artifacts/UnityFsArtifactReader.ts Outdated
Comment thread src/artifacts/UnityFsStructures.ts Outdated
Comment thread src/artifacts/UnityFsArtifactReader.ts
Comment thread src/artifacts/UnityFsArtifactReader.ts Outdated
Comment thread src/artifacts/UnityFsStructures.ts Outdated
Comment thread src/artifacts/DesktopFileInspection.ts Outdated
# Conflicts:
#	src/application/BinaryTargetResolver.ts
#	src/artifacts/createArtifactReader.ts
#	src/artifacts/extraction/ArtifactExtraction.ts
#	src/artifacts/inventory/ArtifactGraphConstruction.ts
#	src/artifacts/inventory/reader.ts
#	src/domain/artifactGraph.ts
#	src/domain/liveBinaryTargetSummary.ts
Comment thread src/artifacts/UnityFsArtifactReader.ts
Comment thread docs/unityfs.md Outdated
@gensokyo-dev
gensokyo-dev merged commit e78ac1a into morluto:main Oct 28, 2026
46 checks passed
Entrpi added a commit to Entrpi/rea that referenced this pull request Oct 28, 2026
… main

Upstream morluto#7699 added the unityfs and unityfs-entry format_metadata kinds.
The fork's exhaustive comparison table must decide each kind, and the fork's
vocabulary test must list every target format. The container's fields come
from its own bytes, so it is not compared. A member declares its source_path
and raw node flags; its index and offset are layout, as for Node SEA members.
The contract docs and the UnityFS guide name the compared fields.
Entrpi added a commit to Entrpi/rea that referenced this pull request Oct 28, 2026
Upstream morluto#7699 added the unityfs target format, so the vocabulary test's
per-format fixtures must name it; without one the test no longer typechecks
against main.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants