Skip to content

feat: prototype the Cardano database snapshot with the Leios database - #3719

Draft
jpraynaud wants to merge 8 commits into
jpraynaud/3687-hard-fork-dijkstra-without-leiosfrom
jpraynaud/3691-hard-fork-dijkstra-with-leios
Draft

jpraynaud wants to merge 8 commits into
jpraynaud/3687-hard-fork-dijkstra-without-leiosfrom
jpraynaud/3691-hard-fork-dijkstra-with-leios

Conversation

@jpraynaud

@jpraynaud jpraynaud commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Content

This PR includes a prototype of the Cardano database snapshot and restoration with the Leios database:

  • Add a Leios mode to the Cardano devnet (USE_LEIOS) with BLS keys for the pools, trustable local roots, short Leios periods and a transaction load that gets endorser blocks certified
  • Add the --use-leios and --leios-load-duration options to the end-to-end tests, which run the scenarios on a Leios devnet
  • Add a leios mode to the end-to-end tests of the CI, running a release candidate of the Leios node
  • Add the mithril-cardano-node-leios crate, which takes a consistent copy of leios.imm.db and checks that it holds the closure of every endorser block certified in the shipped immutable files (temporary implementation)
  • Ship the copy of leios.imm.db in the signed ancillary archive of the aggregator, restored by the client with the other ancillary files (temporary implementation)
  • Check in the end-to-end tests that the restored database holds the closure of every certified endorser block (temporary implementation)

Pre-submit checklist

  • Branch
    • Tests are provided (if possible)
    • Crates versions are updated (if relevant)
    • CHANGELOG file is updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
  • PR
    • All check jobs of the CI have succeeded
    • Self-reviewed the diff
    • Useful pull request description
    • Reviewer requested

Issue(s)

Relates to #3691

@jpraynaud
jpraynaud added this pull request to stack #3720 November 19, 2026 17:22
@jpraynaud
jpraynaud requested a lite review from Copilot November 19, 2026 17:22
@jpraynaud jpraynaud changed the title Jpraynaud/3691 hard fork dijkstra with leios feat: prototype the Cardano database snapshot with the Leios database Nov 19, 2026
@jpraynaud jpraynaud self-assigned this Nov 19, 2026
@jpraynaud jpraynaud added prototype 🛠️ Prototype/PoC of a feature do-not-merge 🔒 Do not merge labels Nov 19, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two active issues remain: the generated Leios configuration omits the required SQLite backend, and the ancillary snapshot check mishandles the final partial immutable chunk, causing normal Leios end-to-end restores to fail.

2 open issues
What changed in this PR

This PR prototypes Leios-aware Cardano database snapshots by configuring a Leios devnet, capturing and restoring leios.imm.db through signed ancillary archives, and validating certified endorser-block closures in end-to-end tests.

Changes:

  • Adds Leios devnet configuration, BLS keys, transaction-load generation, CLI options, documentation, and CI coverage.
  • Introduces mithril-cardano-node-leios for reading certifying blocks, selecting database layouts, and creating consistent SQLite snapshots.
  • Integrates the Leios database into aggregator archives and verifies restored database closures.
File Description
mithril-test-lab/​mithril-end-to-end/​src/​utils/​mithril_command.rs Exposes command working-directory information for end-to-end test utilities.
mithril-test-lab/​mithril-end-to-end/​src/​toolkit/​exec.rs Adds support for executing transaction-load commands.
mithril-test-lab/​mithril-end-to-end/​src/​toolkit/​check/​cardano_database.rs Validates restored Leios database files and certified endorser-block closures.
mithril-test-lab/​mithril-end-to-end/​src/​scenario/​minimal.rs Adds Leios transaction loading and database restoration checks to minimal scenarios.
mithril-test-lab/​mithril-end-to-end/​src/​scenario/​full.rs Adds Leios transaction loading and database restoration checks to full scenarios.
mithril-test-lab/​mithril-end-to-end/​src/​mithril/​infrastructure.rs Propagates Leios-mode configuration through the end-to-end infrastructure.
mithril-test-lab/​mithril-end-to-end/​src/​mithril/​client.rs Locates restored Cardano database v2 files.
mithril-test-lab/​mithril-end-to-end/​src/​main.rs Adds Leios command-line options, defaults, validation, and wiring.
mithril-test-lab/​mithril-end-to-end/​src/​devnet/​cardano.rs Passes Leios settings to the devnet and starts transaction-load generation.
mithril-test-lab/​mithril-end-to-end/​README.md Documents running Leios end-to-end tests.
mithril-test-lab/​mithril-end-to-end/​Cargo.toml Adds the Leios node crate dependency.
mithril-test-lab/​cardano-devnet/​README.md Documents standalone Leios devnet operation.
mithril-test-lab/​cardano-devnet/​mkfiles/​mkfiles-topology-cardano.sh Adds a dedicated transaction-load UTxO address to the topology.
mithril-test-lab/​cardano-devnet/​mkfiles/​mkfiles-start.sh Adds BLS node arguments and transaction-load shutdown handling.
mithril-test-lab/​cardano-devnet/​mkfiles/​mkfiles-leios.sh Generates Leios configuration and transaction-load scripts.
mithril-test-lab/​cardano-devnet/​mkfiles/​mkfiles-init.sh Adds Leios defaults and version validation to devnet initialization.
mithril-test-lab/​cardano-devnet/​mkfiles/​mkfiles-cardano.sh Generates additional UTxO keys and distributes BLS key material.
mithril-test-lab/​cardano-devnet/​mkfiles/​lib/​mkfiles-leios-genesis.sh Generates Leios BLS genesis data and configures Leios parameters.
mithril-test-lab/​cardano-devnet/​devnet-mkfiles.sh Integrates Leios artifact generation into the devnet build.
mithril-aggregator/​src/​services/​snapshotter/​compressed_archive_snapshotter.rs Copies the Leios database into signed ancillary snapshot archives.
mithril-aggregator/​Cargo.toml Adds the Leios node crate dependency to the aggregator.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​test/​mod.rs Provides Leios database and immutable-file test fixtures.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​lib.rs Defines and exports the new Leios crate modules.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​leios_immutable_database_snapshotter.rs Implements consistent Leios database snapshots with retry and closure validation.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​entities/​mod.rs Exports Leios entity types.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​entities/​certifying_block.rs Defines certifying-block metadata and related entities.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​database/​mod.rs Organizes Leios database adapters and layout components.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​database/​leios_immutable_database.rs Provides read-only access and consistent-copy support for the Leios SQLite database.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​database/​layout.rs Defines database layout abstraction and version selection.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​database/​layout_rc_2026w44.rs Implements the rc-2026w44 Leios SQLite database layout.
internal/​cardano-node/​mithril-cardano-node-leios/​src/​certifying_blocks_reader.rs Reads Leios announcements and certifying blocks from immutable files.
internal/​cardano-node/​mithril-cardano-node-leios/​README.md Documents the new crate and its database layout extension model.
internal/​cardano-node/​mithril-cardano-node-leios/​Makefile Adds build, test, check, and documentation targets for the crate.
internal/​cardano-node/​mithril-cardano-node-leios/​Cargo.toml Defines the new workspace crate and its dependencies.
Cargo.toml Registers the Leios crate as a workspace member.
Cargo.lock Records dependencies for the new Leios crate.
.github/​workflows/​ci.yml Adds a Leios-mode end-to-end CI scenario.

Review effort: Lite

…tion and a transaction load

USE_LEIOS runs the devnet in Leios mode with pools able to vote, trustable local roots, short Leios periods, a small block body and a tx-firehose load.
--use-leios runs the devnet in Leios mode with a transaction load, whose duration is set with --leios-load-duration.
@jpraynaud
jpraynaud force-pushed the jpraynaud/3691-hard-fork-dijkstra-with-leios branch from e02bcd5 to 315c055 Compare November 19, 2026 18:02
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Test Results

     5 files  ±  0     215 suites  +4   56m 26s ⏱️ + 1m 21s
 4 454 tests + 39   4 454 ✅ + 39  0 💤 ±0  0 ❌ ±0 
14 515 runs  +144  14 515 ✅ +144  0 💤 ±0  0 ❌ ±0 

Results for commit 75734a5. ± Comparison against base commit 59fb891.

♻️ This comment has been updated with latest results.

@jpraynaud
jpraynaud deployed to testing-preview November 19, 2026 19:08 — with GitHub Actions Active
@jpraynaud
jpraynaud deployed to testing-2-preview November 19, 2026 19:08 — with GitHub Actions Active
Map the `musashinet` network code to the network magic `164`.
Map the `musashinet` network code to the DMQ network magic `2147483812`.
Adds a 'leios' mode to the e2e matrix running the rc-2026w46 Leios node with 15s epochs and a 60s transaction load.
…e of a Cardano node

Takes a consistent copy of leios.imm.db and checks it holds the closure of every endorser block certified in the shipped immutable files.
…ancillary archive

The copy of leios.imm.db is taken after the partial immutable file, checked for the closures of the certifying blocks and added to the signed manifest.
The Cardano database check verifies that the restored database holds the closure of every certified endorser block.
@jpraynaud
jpraynaud force-pushed the jpraynaud/3691-hard-fork-dijkstra-with-leios branch from 315c055 to 75734a5 Compare November 20, 2026 11:12
@jpraynaud
jpraynaud deployed to testing-preview November 20, 2026 11:45 — with GitHub Actions Active
@jpraynaud
jpraynaud deployed to testing-2-preview November 20, 2026 11:45 — with GitHub Actions Active

This branch was successfully deployed

2 active deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge 🔒 Do not merge prototype 🛠️ Prototype/PoC of a feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants