Repository navigation
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Approval is blocked by unresolved critical security and atomicity issues in archive permission sanitization and multi-file promotion, plus a public error-type API gap.
4 open issues
What changed in this PR
This PR strengthens Cardano database archive downloads by safely unpacking untrusted tar archives, validating archive contents before promotion, improving staging and cleanup behavior, and supporting cross-filesystem file moves. It also applies the shared unpacking logic to the client CLI and validates ancillary manifest paths more strictly.
Changes:
- Adds a shared tar unpacker with path, entry-type, and permission controls.
- Adds per-download staging, archive-content validation, cleanup, location fallback, and cross-filesystem promotion.
- Hardens ancillary manifest validation and updates the CLI tar.gz unpacker.
| File | Description |
|---|---|
| mithril-client/src/utils/mod.rs | Exposes ancillary verification helpers needed by download-task tests and logic. |
| mithril-client/src/utils/fs.rs | Adds move-with-cross-filesystem-copy support and related tests. |
| mithril-client/src/utils/ancillary_verifier.rs | Verifies signatures before file hashing and promotes validated ancillary files. |
| mithril-client/src/file_downloader/streaming.rs | Uses the shared tar unpacker for streamed database archives and improves stream and unpack error handling. |
| mithril-client/src/file_downloader/mod.rs | Exports the shared archive unpacker types. |
| mithril-client/src/file_downloader/mock_builder.rs | Adds mock behavior for creating immutable archive contents in staging directories. |
| mithril-client/src/file_downloader/archive_unpacker.rs | Introduces secure tar extraction, path checks, entry filtering, permissions, and extensive tests. |
| mithril-client/src/cardano_database_client/download_unpack/internal_downloader.rs | Validates targets, skips uncompressed locations, manages staging cleanup, and coordinates task shutdown. |
| mithril-client/src/cardano_database_client/download_unpack/download_task.rs | Adds staged downloads, archive validation, promotion, fallback behavior, and cancellation cleanup. |
| mithril-client/src/cardano_database_client/api.rs | Documents the single-download-per-target-directory staging constraint. |
| mithril-client-cli/src/utils/archive_unpacker/tar_gz_unpacker.rs | Replaces the CLI-specific tar unpacking implementation with the shared unpacker. |
| mithril-client-cli/Cargo.toml | Moves the tar dependency to the CLI's development dependencies. |
| internal/cardano-node/mithril-cardano-node-internal-database/src/entities/ancillary_files_manifest.rs | Restricts manifest paths to plain relative paths during deserialization and manifest construction. |
Review effort: Lite
Test Results 5 files ± 0 211 suites ±0 1h 1m 0s ⏱️ - 52m 54s Results for commit cd4ba13. ± Comparison against base commit 1a3c425. This pull request removes 61 and adds 99 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
f6d8495 to
71e74f9
Compare
…h that is not a plain relative path
…he files it names
…d promote only its trio
71e74f9 to
374b74b
Compare
damrobi
left a comment
There was a problem hiding this comment.
LGTM 👍
I left a few small suggestions
Never set the setuid, setgid and sticky bits with fixed unpack permissions. Pin the tests of the staging cleanup, of the stream failure and of the partial copy removal.


Content
This PR includes a strengthening of the download and unpacking of the Cardano database archives in the client and the client CLI:
mithril-client(TarArchiveUnpackerwithUnpackedFilePermissions) that unpacks only the regular files and directories of an archive, and reports any other entry with a typed error0o644for the database files, archived mode without group and other write for the binaries unpacked by the client CLI) and create directories with the default permissionsPre-submit checklist