Skip to content

# Milestone 6: Gossip Message Fuzzing (BOLT 7) #71

Open

Description

@devvaansh

Implements structure-aware fuzzing of BOLT 7 gossip messages — channel_announcement,
node_announcement, channel_update, and announcement_signatures — with valid
secp256k1 signatures, so the fuzzer can penetrate gossip validation logic that
raw-bytes fuzzing cannot reach.

Scope: implements Milestone 6 from the IR Design and Implementation Plan (#5).
Non-goals: routing-table semantics, onion routing, BOLT 11 invoices.

1. Motivation

Every Lightning node accepts gossip from untrusted peers. Bugs in gossip parsing, signature verification, or storage have led to memory exhaustion, CPU DoS, and corrupted routing tables across implementations. The current
EncryptedBytesScenario only stresses early parsing, because raw bytes never satisfy:

  • four chained secp256k1 signatures over a double-SHA256 of the message tail
    (channel_announcement),
  • the lexicographic node_id_1 < node_id_2 ordering rule,
  • chain_hash and short_channel_id consistency between
    channel_announcement → channel_update → announcement_signatures,
  • the must_be_one flag and htlc_minimum_msat ≤ htlc_maximum_msat ≤ capacity
    invariants in channel_update.

On Extending the IR with gossip operations lets the fuzzer construct structurally valid messages whose mutations exercise the deep code paths behind these checks.

2. Current State (baseline)

Area Status
BOLT 7 codecs only gossip_timestamp_filter exists
IR Operations BuildOpenChannel + load/compute primitives only
IR VariableTypes no Signature, ShortChannelId, or Timestamp types
Generators / executor not yet upstream (Milestones 1–5)
msg_type constants gossip types 256–259 missing from bolt.rs

This milestone assumes Milestone 1 (executor + ProgramBuilder + at least one Generator) has landed. It does not depend on Milestones 2–5.

3. High-Level Architecture

Image

4. Deliverables

4.1 BOLT 7 Codecs — smite/smite/src/bolt/

New modules, each exporting an encode / decode pair plus a struct that mirrors the wire layout. All re-exported from bolt.rs
and registered in msg_type:

File Type msg_type
channel_announcement.rs ChannelAnnouncement 256
node_announcement.rs NodeAnnouncement (+ Address enum: ipv4/ipv6/tor‑v3/dns) 257
channel_update.rs ChannelUpdate (+ MessageFlags, ChannelFlags bitfields) 258
announcement_signatures.rs AnnouncementSignatures 259
short_channel_id.rs ShortChannelId (3+3+2 byte packed u64) —

Implementation notes:

  • Reuse the existing WireFormat trait in wire.rs
    (it already supports Signature, PublicKey, and [u8; N]).
  • For ChannelAnnouncement, expose a helper
    signing_hash(&self) -> [u8; 32] that returns the double‑SHA256 of the
    message starting at byte offset 256 (i.e. features onwards). The hash skips
    the four signature slots but covers any trailing bytes, per BOLT 7.
  • For NodeAnnouncement and ChannelUpdate, expose
    signing_hash(&self) -> [u8; 32] covering everything after the leading
    signature field.
  • Round-trip property tests + at least one BOLT‑7 spec vector per message.

4.2 IR Extensions — smite/smite-ir/src/

New VariableTypes (in variable.rs):
Signature, ShortChannelId, Timestamp, plus three compounds for parsed gossip if/when we add Recv* ops (ChannelAnnouncementData, NodeAnnouncementData, ChannelUpdateData) gated behind Extract*
operations following the existing AcceptChannel pattern.

New Operations (in operation.rs):

Load:
  LoadTimestamp(u32)
  LoadShortChannelId(u64)         # packed: (block << 40) | (tx << 16) | out
  LoadRgbColor([u8;3])
  LoadAlias([u8;32])
  LoadAddresses(Vec<u8>)          # already-encoded address descriptor list

Compute (reusable beyond gossip):
  BuildShortChannelId             # inputs: BlockHeight, U16 (tx_index), U16 (out)
                                  # ergonomic alternative to LoadShortChannelId
  DoubleSha256                    # input: Bytes -> Bytes(32)
  Sign                            # inputs: PrivateKey, Bytes(32) -> Signature
                                  # secp256k1 ECDSA, deterministic (RFC 6979)

Build (unsigned + signed pairs, see §4.3):
  BuildChannelAnnouncementUnsigned / BuildChannelAnnouncement
  BuildNodeAnnouncementUnsigned    / BuildNodeAnnouncement
  BuildChannelUpdateUnsigned       / BuildChannelUpdate
  BuildAnnouncementSignatures      # unsigned form not needed (sigs are inputs)

The Sign and DoubleSha256 ops are deliberately generic primitives, they are also prerequisites for future commitment-signature work in Milestones 2–3, so this milestone pays down shared infrastructure debt.

Lexicographic node ordering. BOLT 7 requires node_id_1 < node_id_2. The generator computes both pubkeys at IR-emit time and orders the four LoadPrivateKey instructions accordingly. Subsequent OperationParamMutator
or InputSwapMutator mutations may break the order — that is desirable, since it exercises the receiver's node_id_1 ≥ node_id_2 warning path. No runtime sort op is needed.

4.3 Build Operation Wiring (example — channel_announcement)

Image

Why split unsigned + signed? All four BOLT 7 signed messages sign
the message bytes themselves, with the signature(s) prepended. Encoding the
unsigned form first lets us:

  • feed it directly into a generic DoubleSha256(Bytes) -> Bytes op (no
    message-type-specific hashing logic in the executor);
  • emit each Sign as an independent SSA instruction that any mutator can
    delete, swap, or replace;
  • let OperationParamMutator mutate the unsigned body without invalidating
    signatures the fuzzer wants to keep — and conversely, mutate signatures
    without re-encoding the body. Both halves of that split reach distinct
    validation paths.

Hash regions. The signing region differs per message; the executor stays generic by exposing only DoubleSha256(Bytes), while each codec exposes a signing_region(&self) -> &[u8] helper that the unsigned-Build op uses internally:

Message Bytes hashed
channel_announcement bytes [256..] (skips 4× 64‑byte sig slots)
node_announcement bytes [64..] (skips 1× 64‑byte sig slot)
channel_update bytes [64..] (skips 1× 64‑byte sig slot)
announcement_signatures not signed; signatures come from the chan_ann hash

Concretely each Build*Unsigned op produces a Message variable whose bytes are already in the signing region (i.e. the leading sig slots are zero-filled and stripped before being passed to DoubleSha256). The matching
Build* op accepts the unsigned Message plus the freshly-produced Signature(s) and patches them into the leading slots before SendMessage.

4.4 Generators — smite/smite-ir/src/generators/gossip.rs

Three message generators plus one composing flow generator:

Generator Type Emits
ChannelAnnouncementMsg message 4 keypairs, sort, build unsigned, hash, 4× sign, build signed, SendMessage
NodeAnnouncementMsg message 1 keypair, timestamp, alias/color/addresses, build unsigned, hash, sign, send
ChannelUpdateMsg message scid + chain_hash + flags + fees, build unsigned, hash, sign, send
GossipFlow flow ChannelAnnouncementMsg → 2× ChannelUpdateMsg (one per direction) → 2× NodeAnnouncementMsg. All share the same scid, node_keys, and chain_hash via ProgramBuilder::pick_variable.

Each generator follows the existing pattern: ask ProgramBuilder for typed variables (75 % reuse / 15 % cross-pollinate / 10 % fresh) and emit instructions via builder.append. No generator manipulates indices directly.

AnnouncementSignaturesMsg is gated on the PostChannelOpenSetup snapshot (reuses local_keys / funding_outpoint from ProgramContext) and is
optional for this milestone added only if Milestone 2 has landed.

4.5 Snapshot Setup

Reuse the existing PostInitSetup from Milestone 1. The init message we send
during setup must:

  • advertise gossip_queries (so the target accepts our query stretches if the
    stretch goal is implemented), and
  • immediately follow init with
    GossipTimestampFilter::no_gossip(chain_hash)
    to silence inbound gossip and keep the executor's receive loop predictable.

A new binary entry point, <target>_ir_gossip, is added under
smite-scenarios/src/bin/
wiring IrScenario<T, PostInitSetup> for each of LND / LDK / CLN / Eclair.

Per-target gossip entry symbols the coverage comparison should target
(to be confirmed during implementation):

Target Suspected entry symbols
LND discovery.(*AuthenticatedGossiper).processNetworkAnnouncement, routing.ValidateChannelAnn
LDK lightning::routing::gossip::NetworkGraph::update_channel_from_announcement, update_node_from_announcement
CLN gossipd/queries.c:handle_channel_announcement, routing.c:routing_add_channel_announcement
Eclair fr.acinq.eclair.router.Validation.handleChannelAnnouncement, handleChannelUpdate

4.6 Mutator Interaction

No new mutators required. Existing planned mutators already cover gossip:

Mutator Effect on gossip programs
OperationParamMutator flips bits in LoadFeatures, LoadAlias, LoadTimestamp, LoadShortChannelId
InputSwapMutator swaps node_1 ↔ node_2 keys (breaks ordering → exercises the node_id_1 ≥ node_id_2 warning path)
InstructionDeleteMutator drops the Sign step → exercises bad-signature path
GeneratorInsertionMutator inserts a duplicate BuildChannelUpdate with same timestamp but different fees → exercises the blacklist path

5. Sample Generated Program

v0  = LoadPrivateKey(0x01..)            # node_1 secret
v1  = DerivePoint(v0)
v2  = LoadPrivateKey(0x02..)            # node_2 secret
v3  = DerivePoint(v2)
v4  = LoadPrivateKey(0x03..)            # bitcoin_key_1 secret
v5  = DerivePoint(v4)
v6  = LoadPrivateKey(0x04..)            # bitcoin_key_2 secret
v7  = DerivePoint(v6)
v8  = LoadShortChannelId(0x0837A4_00034D_0001)
v9  = LoadChainHashFromContext()
v10 = LoadFeatures(0x)
v11 = BuildChannelAnnouncementUnsigned(v9, v8, v1, v3, v5, v7, v10)
v12 = DoubleSha256(v11)                 # hashes bytes [256..]
v13 = Sign(v0, v12)
v14 = Sign(v2, v12)
v15 = Sign(v4, v12)
v16 = Sign(v6, v12)
v17 = BuildChannelAnnouncement(v11, v13, v14, v15, v16)
SendMessage(v17)

v18 = LoadTimestamp(1_715_000_000)
v19 = LoadU8(0x01)                      # message_flags: must_be_one
v20 = LoadU8(0x00)                      # channel_flags: direction=node_1, enabled
v21 = LoadU16(144)                      # cltv_expiry_delta
v22 = LoadAmount(1_000)                 # htlc_minimum_msat
v23 = LoadAmount(1_000)                 # fee_base_msat
v24 = LoadAmount(100)                   # fee_proportional_millionths
v25 = LoadAmount(99_000_000)            # htlc_maximum_msat
v26 = BuildChannelUpdateUnsigned(v9, v8, v18, v19, v20, v21, v22, v23, v24, v25)
v27 = DoubleSha256(v26)
v28 = Sign(v0, v27)
v29 = BuildChannelUpdate(v26, v28)
SendMessage(v29)

6. Validation & Coverage Analysis

Unit tests

  • Round-trip codec tests for all four messages (encode → decode → encode).
  • BOLT 7 spec vectors where available (mainnet chain_hash, ordering rule).
  • Generator tests asserting node_id_1 < node_id_2 and matching scid /
    chain_hash between flow-generated channel_announcement and its two
    channel_updates.

End-to-end fuzzing

  • 24 h fuzzing runs on all four supported targets using the existing
    coverage-report.sh workflow.
  • Acceptance criterion: ir_gossip corpus must hit functions named
    *verify_channel_announcement*, *process_channel_update*,
    *store_node_announcement* (or per-target equivalents) that the
    encrypted_bytes corpus does not reach. Comparison published as a small
    table in the PR description.

7. Stretch Goals

  1. Gossip queries — add codecs + Build* ops for query_channel_range,
    reply_channel_range (incl. timestamps_tlv / checksums_tlv with
    CRC32C), query_short_channel_ids, and reply_short_channel_ids_end.
    These are unsigned, so they reuse Sign infrastructure trivially.
  2. Plausible funding outpoints — add a MineFundingTx action generator
    that, before emitting a channel_announcement, drives bitcoind (already
    available to the executor) to mine a transaction with a P2WSH output spending
    to 2-of-2 multisig(bitcoin_key_1, bitcoin_key_2). The resulting
    (block, tx_index, vout) is fed into BuildShortChannelId, allowing the
    message to pass on-chain validation in targets that perform UTXO lookup
    (e.g. CLN with --funding-confirms-required).
  3. Regtest splice announcements — once Milestone 5 lands, exercise the
    splice-driven channel_announcement regeneration path.

8. Risks & Open Questions

  • secp256k1 signing on the VM hot path. Each Sign op costs ~80 µs on
    modern x86. A GossipFlow runs ≤ 8 signatures (4 for chan_ann +
    2× chan_update + 2× node_ann) ≈ 0.6 ms — well below the dominant Nyx
    snapshot-restore cost. The unsigned/signed split (§4.3) means mutators
    don't need to re-sign on every byte flip, so the steady-state cost
    stays low.
  • Compound Recv* types deferred. Gossip is largely a one-way ingress
    surface; the target rarely replies synchronously. The executor's existing
    auto-pong loop already drains anything inbound. If/when a gossip oracle is
    added (e.g. "target must not re-broadcast a chan_ann with bad sigs"), we
    can add RecvChannelAnnouncement + Extract* following the
    AcceptChannel pattern.
  • short_channel_id realism. Without the §8 funding-outpoint stretch
    goal, generated SCIDs reference non-existent UTXOs. CLN and LND will reject
    channel_announcement early on UTXO lookup; LDK and Eclair are less strict
    in default config. To guarantee all four targets reach signature
    verification, the ir_gossip workload Dockerfiles disable on-chain UTXO
    validation where possible (e.g. CLN --dev-fast-gossip --dev-allow-localhost,
    LND regtest with funding-validation off). Each per-target tweak is documented
    in the corresponding workloads/<target>/Dockerfile change.
  • Spec ambiguity around must_be_one. Receivers are now required to
    ignore it, but several implementations historically rejected
    message_flags & 0x01 == 0. Exposed as a plain LoadU8 so the fuzzer can
    flip it.

Activity

  1. morehouse commented on May 11, 2026

    @morehouse
    Collaborator

    Concept ACK.

    The approach mostly looks good. I added some comments/questions/suggestions inline.

    3. High-Level Architecture

    Image

    What's up with the "Core IR Types" section? It is empty and weirdly shaped.

    4.2 IR Extensions — smite/smite-ir/src/

    New VariableTypes (in variable.rs): Signature, ShortChannelId, Timestamp, plus three compounds for parsed gossip if/when we add Recv* ops (ChannelAnnouncementData, NodeAnnouncementData, ChannelUpdateData) gated behind Extract* operations following the existing AcceptChannel pattern.

    What type(s) do you intend to use for fee_base_msat and fee_proportional_millionths? I think it may be worth having a u32 Fee type.

    I also wonder if we really need a Signature type, or if sigs should be computed internally instead (see below).

    New Operations (in operation.rs):

    Load:
      LoadTimestamp(u32)
      LoadShortChannelId(u64)         # packed: (block << 40) | (tx << 16) | out
      LoadRgbColor([u8;3])
      LoadAlias([u8;32])
      LoadAddresses(Vec<u8>)          # already-encoded address descriptor list
    
    Compute (reusable beyond gossip):
      BuildShortChannelId             # inputs: BlockHeight, U16 (tx_index), U16 (out)
                                      # ergonomic alternative to LoadShortChannelId
      DoubleSha256                    # input: Bytes -> Bytes(32)
      Sign                            # inputs: PrivateKey, Bytes(32) -> Signature
                                      # secp256k1 ECDSA, deterministic (RFC 6979)
    
    Build (unsigned + signed pairs, see §4.3):
      BuildChannelAnnouncementUnsigned / BuildChannelAnnouncement
      BuildNodeAnnouncementUnsigned    / BuildNodeAnnouncement
      BuildChannelUpdateUnsigned       / BuildChannelUpdate
      BuildAnnouncementSignatures      # unsigned form not needed (sigs are inputs)
    

    We could simplify this quite a bit -- we could get rid of DoubleSha256 and Sign and all the Build*Unsigned operations, instead computing hashes and signatures directly in the Build* operations. The only thing we would lose is the ability to swap signatures or generate invalid signatures, but I'm not sure that those are going to be as useful of paths to test anyway. And if we do want to later test the invalid-signature path better, we could add some sort of raw-bytes mutator that could easily invalidate signatures.

    I'm also not sure whether we'd actually use both LoadShortChannelId and BuildShortChannelId. I think initially we should only implement the one we actually use. We can always extend it later if we have a clear use case.

    Finally, what should the output type be for LoadRgbColor? Do we need a new variable type?

    Lexicographic node ordering. BOLT 7 requires node_id_1 < node_id_2. The generator computes both pubkeys at IR-emit time and orders the four LoadPrivateKey instructions accordingly. Subsequent OperationParamMutator or InputSwapMutator mutations may break the order — that is desirable, since it exercises the receiver's node_id_1 ≥ node_id_2 warning path. No runtime sort op is needed.

    The sort could also easily be done when executing the Build* operations. But I also question if we really need to do the sort. We could just put the signatures in the same order as the key params -- we'd still get the correct order 50% of the time. I'd probably start with that, then if we determine that key order is a significant bottleneck we could always add the sort later.

    Why split unsigned + signed? All four BOLT 7 signed messages sign the message bytes themselves, with the signature(s) prepended. Encoding the unsigned form first lets us:

    * feed it directly into a generic `DoubleSha256(Bytes) -> Bytes` op (no
      message-type-specific hashing logic in the executor);
    
    * emit each `Sign` as an independent SSA instruction that any mutator can
      delete, swap, or replace;
    
    * let `OperationParamMutator` mutate the unsigned body _without invalidating_
      signatures the fuzzer wants to keep — and conversely, mutate signatures
      without re-encoding the body. Both halves of that split reach distinct
      validation paths.
    

    I'm not convinced any of these are good reasons to split unsigned/signed operations. I don't think having distinct DoubleSha256 and Sign operations actually get us much practical benefit -- any mutations of the hash or signatures will invalidate them and prevent deeper fuzzing. Also I don't understand the third point -- if we mutate the message body, the signatures automatically get invalidated (that's the whole point of the signature), so I don't see why we would want the ability to do that.

    Three message generators plus one composing flow generator:
    Generator Type Emits
    ChannelAnnouncementMsg message 4 keypairs, sort, build unsigned, hash, 4× sign, build signed, SendMessage
    NodeAnnouncementMsg message 1 keypair, timestamp, alias/color/addresses, build unsigned, hash, sign, send
    ChannelUpdateMsg message scid + chain_hash + flags + fees, build unsigned, hash, sign, send
    GossipFlow flow ChannelAnnouncementMsg → 2× ChannelUpdateMsg (one per direction) → 2× NodeAnnouncementMsg. All share the same scid, node_keys, and chain_hash via ProgramBuilder::pick_variable.

    I'm not 100% sure we need the GossipFlow generator. Perhaps we could start with the other 3 generators, measure coverage, and then add GossipFlow and see if it provides measurable benefit.

    A new binary entry point, <target>_ir_gossip, is added under smite-scenarios/src/bin/ wiring IrScenario<T, PostInitSetup> for each of LND / LDK / CLN / Eclair.

    If we're reusing/extending PostInitSetup, then we really don't need a new binary. But we'll need to make sure that the no_gossip filter is enough to prevent gossip noise.

    * **`short_channel_id` realism.** Without the §8 funding-outpoint stretch
      goal, generated SCIDs reference non-existent UTXOs. CLN and LND will reject
      `channel_announcement` early on UTXO lookup; LDK and Eclair are less strict
      in default config. To guarantee all four targets reach signature
      verification, the `ir_gossip` workload Dockerfiles disable on-chain UTXO
      validation where possible (e.g. CLN `--dev-fast-gossip --dev-allow-localhost`,
      LND regtest with funding-validation off). Each per-target tweak is documented
      in the corresponding `workloads/<target>/Dockerfile` change.
    

    I think I'd prefer to avoid changing the dockerfiles for this. I'm fine with CLN/LDK not being fuzzed as deeply until we merge mining support.


    I want to fast-track this work so that @Chand-ra has more diversity of IR programs to test his new mutators on (e.g., #61). So I'll do my best to review the PRs quickly, and I'll probably also contribute some of the PRs myself when I have free cycles.

    We should make sure to post what we will be working on here so that we don't step on each others toes.

  2. devvaansh commented on May 13, 2026

    @devvaansh
    ContributorAuthor

    What's up with the "Core IR Types" section? It is empty and weirdly shaped.

    That's a mermaid rendering bug on my end -- the outer subgraph was meant to contain the inner ones, not sit beside them
    flowchart LR

    I'll edit the issue body once we converge on the rest.

    What type(s) do you intend to use for fee_base_msat and fee_proportional_millionths? I think it may be worth having a u32 Fee type.
    I also wonder if we really need a Signature type, or if sigs should be computed internally instead (see below).

    Agree on both:

    I'll add a Fee(u32) variable type (millisatoshis for fee_base_msat, ppm for fee_proportional_millionths). Reusing Amount would have been wrong — Amount is u64 satoshis in the existing code, and BOLT 7 explicitly bounds these to u32.
    Given your point below about folding signing into Build*, drop Signature as a public variable type. Signatures only ever appear as transient intermediates inside the executor, never as SSA values the mutator can reach.

    We could simplify this quite a bit -- we could get rid of DoubleSha256 and Sign and all the BuildUnsigned operations, instead computing hashes and signatures directly in the Build operations. The only thing we would lose is the ability to swap signatures or generate invalid signatures, but I'm not sure that those are going to be as useful of paths to test anyway. And if we do want to later test the invalid-signature path better, we could add some sort of raw-bytes mutator that could easily invalidate signatures.

    Agree, you've convinced me. My justification for the split was mostly "let mutators target signatures independently of the body," but you're right that:

    mutating the body invalidates the sigs anyway, so the "preserve sigs while mutating body" path doesn't exist in practice;
    mutating just the sig bytes is exactly what a future raw-bytes / post-encoding mutator would do far more cheaply than reshuffling SSA.
    removing DoubleSha256/Sign deletes an entire class of "mutator produces an op graph that doesn't actually sign the right bytes" failure modes that would otherwise eat fuzzer cycles on uninteresting paths.
    So the revised op surface is just:

    Load: LoadTimestamp(u32), LoadShortChannelId(u64),
    LoadRgbColor(...), LoadAlias([u8;32]), LoadAddresses(Vec),
    LoadFee(u32)

    Build: BuildChannelAnnouncement, BuildNodeAnnouncement,
    BuildChannelUpdate, BuildAnnouncementSignatures

    No Compute additions at all for this milestone. Hashing/signing happens inside each Build* executor, using the per-codec signing_region() helper I already added in [channel_update.rs].

    I'm also not sure whether we'd actually use both LoadShortChannelId and BuildShortChannelId. I think initially we should only implement the one we actually use. We can always extend it later if we have a clear use case.

    Agree. Keep only LoadShortChannelId(u64). Rationale: until the mining-support stretch goal lands, generators have no real (block, tx_index, vout) to feed BuildShortChannelId with anyway -- they'd just be loading three constants and packing them, which is what Load does in one step. We can add it the day there's a real BlockHeight SSA value to consume.

    Finally, what should the output type be for LoadRgbColor? Do we need a new variable type?

    Honest answer: I don't think it warrants a dedicated RgbColor variable type -- there are no other ops that consume "an RGB color." Two options, your call:

    (a) Reuse Bytes and have the executor reject anything not exactly 3 bytes. Cheap, but loses static guarantees.

    (b) Inline it as an immediate parameter on BuildNodeAnnouncement(rgb_color: [u8; 3]) instead of a separate Load* op. We lose mutator reach on the color field, but the field is 3 bytes of cosmetic metadata that doesn't gate any validation path I'm aware of , almost certainly not worth a Variable enum slot.

    I lean (b). Worth surfacing as Load* only if you think the mutator should be able to point multiple BuildNodeAnnouncement ops at the same color value.

    The sort could also easily be done when executing the Build* operations. But I also question if we really need to do the sort. We could just put the signatures in the same order as the key params -- we'd still get the correct order 50% of the time. I'd probably start with that, then if we determine that key order is a significant bottleneck we could always add the sort later.

    Agree-- start with "emit signatures in the same order as the keys, accept ~50% wrong order." That's actually better fuzzing input diversity than always-sorted, since every program exercises one of the two ordering branches. If we later see the receiver's node_id_1 ≥ node_id_2 rejection path dominating coverage, we add the sort.

    I'm not convinced any of these are good reasons to split unsigned/signed operations. I don't think having distinct DoubleSha256 and Sign operations actually get us much practical benefit -- any mutations of the hash or signatures will invalidate them and prevent deeper fuzzing. Also I don't understand the third point -- if we mutate the message body, the signatures automatically get invalidated (that's the whole point of the signature), so I don't see why we would want the ability >to do that.

    You're right on all three -- I was overthinking it.

    Walking through my own bullets after re-reading them:

    Generic DoubleSha256 op — the "genericness" only pays off if something other than gossip signing consumes it. Nothing in the near-term roadmap does. Folding the hash into each Build* executor is strictly less code and removes a class of "op graph that hashes the wrong byte range" failures that would just burn fuzzer cycles.

    Sign as independent SSA so mutators can delete/swap/replace it -- InstructionDeleteMutator dropping the Sign produces an unsigned message the target rejects at the first signature check. That's one shallow path, and a future raw-bytes mutator (as you suggested) can hit it more cheaply by zeroing the sig field post-encoding. Not worth a dedicated op.

    Mutate body without invalidating sigs -- you're correct, this is incoherent. I was imagining a scenario where the mutator tweaks one body field and the executor re-signs, but that's just "mutate the body" with a single Build* op, the unsigned/signed split doesn't enable anything there. Wrong mental model on my part.

    So: dropping Build*Unsigned, DoubleSha256, and Sign. Each Build* op takes the keys + payload fields directly and produces a fully-signed Message variable. If we later want richer invalid-signature coverage, a raw-bytes post-encoding mutator is the right tool -- agreed.

    I'm not 100% sure we need the GossipFlow generator. Perhaps we could start with the other 3 generators, measure coverage, and then add GossipFlow and see if it provides measurable benefit.

    Agree. Ship the three message generators, run the 24 h coverage diff against encrypted_bytes, then decide based on data whether GossipFlow's shared-scid cross-message paths add anything pick_variable reuse doesn't already give us for free.

    If we're reusing/extending PostInitSetup, then we really don't need a new binary. But we'll need to make sure that the no_gossip filter is enough to prevent gossip noise.

    Agree, drop the _ir_gossip binary. I checked each target's gossip-filter handling so we can frame the verification work concretely:

    LND: discovery/gossiper.go honors a remote peer's gossip_timestamp_range for historical replay, but I want to verify it also gates future broadcasts to us. Quick smoke test on regtest will confirm.

    LDK: peer_handler.rs tracks sent_gossip_timestamp_filter and stops dispatching gossip once we send a filter; no_gossip (first_timestamp = u32::MAX, timestamp_range = 0) should be honored cleanly.

    CLN: gossipd is the relevant daemon. I'll verify the timestamp filter actually short-circuits the per-peer broadcast loop and not just the historical query response.

    Eclair: router/Sync.scala sends and respects GossipTimestampFilter in the sync flow; same expectation.
    If any target ignores the filter I'll add a narrow per-target receive-side drain rather than a new binary.

    I think I'd prefer to avoid changing the dockerfiles for this. I'm fine with CLN/LDK not being fuzzed as deeply until we merge mining support.

    Agree. I'll drop the Dockerfile changes from scope and add a one-line note to the milestone success criteria that "deep CLN / LND coverage is gated on mining support landing." LDK + Eclair coverage is still a meaningful first deliverable.

  3. devvaansh commented on May 14, 2026

    @devvaansh
    ContributorAuthor

    I want to fast-track this work so that @Chand-ra has more diversity of IR programs to test his new mutators on (e.g., #61). So I'll do my best to review the PRs quickly, and I'll probably also contribute some of the PRs myself when I have free cycles.
    We should make sure to post what we will be working on here so that we don't step on each others toes.

    Sounds good.

    Proposed split

    PR Scope Owner
    #A BOLT 7 codecs short_channel_id, channel_announcement, node_announcement, channel_update, announcement_signatures + msg_type constants. No IR changes. me
    #B IR variable types Fee(u32), ShortChannelId, Timestamp in smite-ir/src/variable.rs me
    #C Load* ops LoadTimestamp, LoadShortChannelId, LoadAlias, LoadAddresses, LoadFee me
    #D Build* ops + executor BuildChannelAnnouncement / BuildNodeAnnouncement / BuildChannelUpdate / BuildAnnouncementSignatures with hash+sign inline open — happy to take, defer to you if you'd rather
    #E Generators ChannelAnnouncementMsg, NodeAnnouncementMsg, ChannelUpdateMsg open
    #F Coverage report 24h run on LDK + Eclair, per-target table me, after #E

    I'll start with #A (no IR dependencies), then move on to #B and #C in parallel since they're also independent. Will post here whenever I land a PR so you and Chand-ra always know the live state. If you'd rather grab #D/#E to keep them moving while I'm working through codecs, just say the word otherwise I'll pick them up after #C.

    One coordination ask: if you do end up taking any of these yourself, a quick comment on this issue before you start would let me re-route. I'll do the same.

  4. morehouse commented on May 14, 2026

    @morehouse
    Collaborator

    Honest answer: I don't think it warrants a dedicated RgbColor variable type -- there are no other ops that consume "an RGB color." Two options, your call:

    (a) Reuse Bytes and have the executor reject anything not exactly 3 bytes. Cheap, but loses static guarantees.

    (b) Inline it as an immediate parameter on BuildNodeAnnouncement(rgb_color: [u8; 3]) instead of a separate Load* op. We lose mutator reach on the color field, but the field is 3 bytes of cosmetic metadata that doesn't gate any validation path I'm aware of , almost certainly not worth a Variable enum slot.

    I lean (b). Worth surfacing as Load* only if you think the mutator should be able to point multiple BuildNodeAnnouncement ops at the same color value.

    Yep, I think (b) is the way to go.

    Proposed split
    PR Scope Owner
    #A BOLT 7 codecs short_channel_id, channel_announcement, node_announcement, channel_update, announcement_signatures + msg_type constants. No IR changes. me
    #B IR variable types Fee(u32), ShortChannelId, Timestamp in smite-ir/src/variable.rs me
    #C Load* ops LoadTimestamp, LoadShortChannelId, LoadAlias, LoadAddresses, LoadFee me
    #D Build* ops + executor BuildChannelAnnouncement / BuildNodeAnnouncement / BuildChannelUpdate / BuildAnnouncementSignatures with hash+sign inline open — happy to take, defer to you if you'd rather
    #E Generators ChannelAnnouncementMsg, NodeAnnouncementMsg, ChannelUpdateMsg open
    #F Coverage report 24h run on LDK + Eclair, per-target table me, after #E

    I don't want to wait to contribute until A, B, and C are all done. I propose that we split the work vertically instead of horizontally, so we can work concurrently.

    My plan is to focus on NodeAnnouncementGenerator first, along with everything it depends on (node_announcement codec, Timestamp variable type, LoadTimestamp, BuildNodeAnnouncement). You could choose another vertical slice to work on, or you could work horizontally as you suggested, skipping the parts I mentioned that relate to node_announcement.

  5. devvaansh commented on May 14, 2026

    @devvaansh
    ContributorAuthor

    I don't want to wait to contribute until A, B, and C are all done. I propose that we split the work vertically instead of horizontally, so we can work concurrently.
    My plan is to focus on NodeAnnouncementGenerator first, along with everything it depends on (node_announcement codec, Timestamp variable type, LoadTimestamp, BuildNodeAnnouncement). You could choose another vertical slice to work on, or you could work horizontally as you suggested, skipping the parts I mentioned that relate to node_announcement.

    Agreed on vertical slices, much better than my horizontal proposal, gets parallelism going immediately.

    I'll take the channel_update vertical: channel_update codec, short_channel_id codec + helpers, Fee(u32) + ShortChannelId variable types, LoadShortChannelId + LoadFee ops, BuildChannelUpdate (hash+sign inline), and ChannelUpdateGenerator. The two codecs are already done on gossip-codecs-bolt7 so I can open that PR today and chase the IR-side pieces from there.

    That leaves the channel_announcement + announcement_signatures vertical unclaimed (codecs + BuildChannelAnnouncement / BuildAnnouncementSignatures + ChannelAnnouncementGenerator). I'm happy to pick that up after the channel_update vertical lands, unless you or someone else wants it sooner, say the word and I'll defer.

    LoadAlias / LoadAddresses fall naturally into your node_announcement vertical, so I won't touch those.

    Refreshed table:

    Vertical Scope Owner
    node_announcement node_announcement codec, Timestamp type, LoadTimestamp, LoadAlias, LoadAddresses, BuildNodeAnnouncement (rgb_color inlined), NodeAnnouncementGenerator Matt
    channel_update channel_update + short_channel_id codecs, Fee(u32) + ShortChannelId types, LoadShortChannelId, LoadFee, BuildChannelUpdate, ChannelUpdateGenerator me
    channel_announcement channel_announcement + announcement_signatures codecs, BuildChannelAnnouncement, BuildAnnouncementSignatures, ChannelAnnouncementGenerator open — I'll take after channel_update lands unless claimed
    coverage report 24h run on LDK + Eclair, per-target diff vs encrypted_bytes me, after all 3 verticals land

    will push the codec pr today

  6. morehouse commented on May 21, 2026

    @morehouse
    Collaborator

    I've completed the initial node_announcement generator, pending code review. Starting to work on channel_announcement now.

  7. devvaansh commented on May 24, 2026

    @devvaansh
    ContributorAuthor

    I've completed the initial node_announcement generator, pending code review. Starting to work on channel_announcement now.

    I've been tied up with university exams. Back now, just opened #96 (LoadShortChannelId) as the first IR
    piece of the channel_update vertical. I’ll continue with BuildChannelUpdate next.

  8. devvaansh commented on Jun 22, 2026

    @devvaansh
    ContributorAuthor

    @morehouse , I Ran both workloads for 24h each, concurrently, on LDK (ldk-node-wrapper, AFL++ 5.02a in Nyx snapshot mode, smite e4f488b), then diffed per-function LLVM source-based coverage. No crashes or hangs on either side. Full report, reproduction, and the complete 680-function list: https://gist.github.com/devvaansh/edc80b0f255d849675a93e606c09a10d

    Where the two workloads diverge

    Random bytes occasionally parse as a gossip message and therefore reach LDK's signature-verification functions — but they carry random 64-byte signatures, so they always fail. Only ir produces validly-signed gossip that passes verification and mutates the routing graph.

    flowchart TD
        A["raw input bytes"] --> B["wire decode:<br/>ChannelAnnouncement / NodeAnnouncement / ChannelUpdate"]
        B --> C["RoutingMessageHandler::handle_*"]
        C --> D["verify_channel_announcement / verify_node_announcement<br/>message_sha256d_hash + secp256k1"]
        D -->|"signature INVALID"| X["reject — return error<br/>(encrypted_bytes stops here)"]
        D -->|"signature VALID"| E["update_channel_from_unsigned_announcement_intern<br/>update_node_from_announcement_intern"]
        E --> F["NetworkGraph mutation:<br/>NodeId→NodeInfo / channel maps + UTXO resolution"]
        classDef both fill:#d4edda,stroke:#28a745,color:#111;
        classDef ironly fill:#cce5ff,stroke:#004085,color:#111;
        classDef reject fill:#f8d7da,stroke:#721c24,color:#111;
        class A,B,C,D both;
        class X reject;
        class E,F ironly;
    
    Loading
    LDK function encrypted_bytes ir
    handle_channel_announcement / handle_node_announcement / handle_channel_update ✅ ✅
    verify_channel_announcement / verify_node_announcement / message_sha256d_hash ✅ ✅
    update_channel_from_announcement / update_node_from_announcement / update_channel_internal ✅ ✅
    update_channel_from_unsigned_announcement_intern (post-verify graph insert) ❌ ✅
    update_node_from_announcement_intern (post-verify graph insert) ❌ ✅
    The split is the signature gate: both reach verify_*; only ir reaches the *_intern graph-insertion functions that run after a signature passes — and the ~680-function downstream subtree (NetworkGraph mutation + the routing::utxo resolution path) that only executes once gossip is accepted.

    Coverage (ldk-node-wrapper, all crates, demangled & de-duplicated)

    functions hit line function region
    encrypted_bytes 5,003 9.02% 9.73% 9.28%
    ir 5,359 9.53% 10.34% 9.68%
    reached only by ir 680

    Campaign (AFL++ fuzzer_stats, 23.8h each)

    ir encrypted_bytes
    execs 42,247,966 47,823,962
    corpus 304 2,560
    cycles_done 191 49
    crashes / hangs 0 / 0 0 / 0
    ir keeps a small corpus of deep, validly-structured programs; encrypted_bytes accumulates many shallow raw-byte inputs. Both reached steady state before 24h.

    Caveats (honesty)

    • encrypted_bytes does reach the verify_* functions (random bytes sometimes parse as gossip and call verification, then fail it). The real delta is the post-verification *_intern graph-insertion path and its downstream subtree — not the verify functions themselves.
    • Single target (LDK) this run. CLN/LND reject earlier on UTXO lookup until mining support lands (deferred per this thread); Eclair can be added with the same harness.
    • Fuzz-time edge stability was 36.5% (ir) / 62.0% (eb) — the LDK-node target is stateful/async. The coverage report replays the full saved corpus through a deterministic source-coverage build and unions reached functions, so the function-reach comparison is robust to fuzz-time flakiness.
      Browsable HTML coverage and full method/repro are in the gist.

  9. morehouse commented on Jun 24, 2026

    @morehouse
    Collaborator

    @devvaansh: looks good.

    Now the main blocker for hitting deeper gossip code is the on-chain validation checks. To get past those, we need to implement the ability to mine 2-of-2 P2WSH outputs composed from two funding keys. We then need to create the channel_announcement message using the funding keys and the SCID derived from the funding output.

  10. devvaansh commented on Jun 25, 2026

    @devvaansh
    ContributorAuthor

    @devvaansh: looks good.
    Now the main blocker for hitting deeper gossip code is the on-chain validation checks. To get past those, we need to implement the ability to mine 2-of-2 P2WSH outputs composed from two funding keys. We then need to create the channel_announcement message using the funding keys and the SCID derived from the funding output.

    Thanks. This is the "Plausible funding outpoints" stretch goal from §7.

    Good news, most of the plumbing is already there: BitcoinCli, CreateFundingTransaction, BroadcastTransaction, and MineBlocks are all upstream. What's missing is the gossip-specific bridge:

    2-of-2 P2WSH funding tx — CreateFundingTransaction already builds the channel funding output from two pubkeys; needs a small variant (or a new CreateP2wsh2of2FundingTransaction) that emits a transaction we can confirm and later reference by SCID
    Capture the confirmed outpoint, once MineBlocks confirms it, derive (block_height, tx_index, vout) and surface it as a ShortChannelId variable (likely a LoadShortChannelIdFromConfirmedTx op, or extend CreateFundingTransaction to also output a ShortChannelId)

    Wire into ChannelAnnouncementGenerator — replace the current LoadShortChannelId(random) + dummy keys path with: pick or create two bitcoin keys → emit the funding tx → mine → use the derived SCID and those same keys in BuildChannelAnnouncement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions