Repository navigation
[softnpu, mcast] wire up the softnpu backend for multicast table ops - #405
Open
zeeshanlakhani wants to merge 28 commits into
Open
zeeshanlakhani wants to merge 28 commits into
zeeshanlakhani wants to merge 28 commits into
Conversation
Previously, SoftNPU didn't interact with multicast. `mc_group_create` and `mc_port_add` returned a stubbed `OperationUnsupported`, and port remove/destroy were no-ops. With bitmap-based replication in p4rs ([#240](oxidecomputer/p4#240)), SoftNPU can replicate off of port bitmaps carried in the replication table entry itself. On the Tofino, membership subscriptions live in the PRE and change independently of match-action state. In the SoftNPU work, membership is captured at write time, and a membership mutation means rewriting the affected replication entries (when triggered). Includes: - *asic/SoftNPU*: - Live replication entries are tracked per group and rewritten when members are added/removed. The replication tracking lock covers both the snapshot and table updates. - Source filter, mcast NAT (tagged and untagged), egress decap, and mcast src MAC tables now map onto their sidecar-lite tables respectively. - `egress.asic_id_to_port` maps ASIC port IDs to logical port numbers for indexing the decap bitmap (same as on Tofino). - `TableRemove` is sent before `TableAdd` because p4rs drops an add whose key already exists and keeps the older entry (action and params). - *dpd/mcast*: - `mcast::reset` deletes each known group's entries before the table clears. SoftNPU's `Table::clear` is still a stub. - `del_entries_where` skips the stale-entry table sweep when the backend cannot list entries (e.g., in SoftNPU). - softnpu-lib is pinned to `zl/multicast` @ softnpu#183, which pulls in p4#240. SoftNPU loads sidecar-lite#152 at runtime.
zeeshanlakhani
requested review from
FelixMcFelix,
cfzimmerman,
rcgoodfellow and
taspelund
October 21, 2026 02:37
zeeshanlakhani
added a commit
to oxidecomputer/sidecar-lite
that referenced
this pull request
Oct 26, 2026
This adds multicast replication to sidecar-lite, emulating DPD's sidecar.p4 on dendrite up through oxidecomputer/dendrite#405 following RFD 488. We manage 2 bitmaps per underlay group (external and underlay), for which replication is driven by p4rs `replicate` (oxidecomputer/p4#240; we track p4 on main now). Ingress zeroes out at most one of these bitmaps determined by the geneve multicast option tag (`tag=0` zeroes out underlay; `tag=1` zeroes out external), and merges both. Tag 0 (External) also decaps upon ingress. With no tag given, as on external NAT ingress packets, both bitmaps remain in place. Only geneve-encapsulated multicast is replicated. Multicast packets that miss ingress NAT drop on the empty bitmap, matching DPD's `drop_mcastv{4,6}_no_group` semantics. Packet copies that remain encapsulated lose a hop on the outer IPv6 header, as modelled in `MulticastRouter6`. External multicast is NAT-encapsulated into the underlay via `nat_v{4,6}_mcast(_untagged)`, reusing `forward_to_sled`. Tagged and untagged variants exist as separate tables since x4c cannot key directly on header validity. The encap piece now zeroes out the outer src, traffic class, flow label, and inner eth src, like DPD's `add_encap_headers`. Egress decap checks `ff04::/64` and `tag=2`, matching DPD's egress `mcast_tag_check`, with per-port decap from a RID-keyed bitmap (`tbl_decap_ports`) and optional VLAN. `tag=2` decapsulation also rejects inner unicast. `tag=0` copies (already decapped in ingress) use the table for the VLAN. Dst MAC derivation (RFC 1112, RFC 2464) and per-port src MAC rewriting run on every packet copy. Includes: - The parser rejects `ff01` addresses outright (`ff02` *does* still go to the scrimlet). - Multicast packets with a TTL/hop_limit <= 1 get dropped in ingress after the `fwd_from_scrimlet` return, letting scrimlet transmissions pass through. - Both decapsulation paths drop inner TTL <= 1. IPv4 checksums use RFC 1624 (eqn. 3): decrementing the TTL adds 0xfeff in one's complement, which yields 0x0000 rather than 0xffff at the zero boundary, matching a full recompute (XOR is used here due to `~` not being parsable by x4c). - scadm gets add/remove/dump for replication (v6), source filters, mcast NAT, decap ports, src MAC, and asic-id-to-port (for logical mapping). - Unicast modifications: - The router drops TTL/hop_limit <= 1 and decrements remaining packets on routes to nonzero ports, updating the IPv4 checksum. - Port 0 (the scrimlet) skips the `<= 1` check and the decrement, while a TTL/hop_limit `= 0` still drops before route lookup, except on the boundary services decapsulation path. - DPD's Router4/6 still decrement service-bound packets. - The NAT encap zeroing described above is shared with unicast NAT. - Packets with a parsed Geneve header skip ingress NAT, including unicast.
cfzimmerman
reviewed
Oct 28, 2026
cfzimmerman
left a comment
Contributor
There was a problem hiding this comment.
Voxel exercises softnpu, right? Can you run a voxel test and post some validation results? I'll stamp once we know it works.
Contributor
Author
Incoming with this line of work. Via oxidecomputer/voxel#75 and past iterations, I demo'ed the prototype set of branches working. |
cfzimmerman
approved these changes
Oct 30, 2026
cfzimmerman
left a comment
Contributor
There was a problem hiding this comment.
Left a reply on the table.rs diff. Otherwise looks good to me once we're confident in e2e voxel testing.
zeeshanlakhani
force-pushed
the
zl/mcast-softnpu
branch
from
November 2, 2026 16:34
70ff3bc to
3019e6f
Compare
This also replaces ``make_tag(TEST_TAG)` w/ `TEST_TAG`` in two dpd-client mcast tests.
zeeshanlakhani
force-pushed
the
zl/mcast-softnpu
branch
from
November 6, 2026 06:11
3562104 to
50c7013
Compare
rcgoodfellow
reviewed
Nov 6, 2026
On a PUT to an mcast group awaiting repair, we now rewrite the egress decap bitmap from its stored membership before modifying membership (in the case that rollback may have dropped it). Underlay cleanup now deletes the replication entry first and then maintains the decap bitmap if the delete fails. Decapsulation holds a superset of all the ports replication can still send to.
Note on *deps*: - softnpu -> zl/table-clear (oxidecomputer/softnpu#234) - p4 -> zl/upsert-table-entries (oxidecomputer/p4#282)
zeeshanlakhani
added a commit
to oxidecomputer/p4
that referenced
this pull request
Nov 10, 2026
…282) `HashSet::insert` retained the prev entry, which meant that adding the same key with a new action or new params was a no-op. We now use replace per @rcgoodfellow @ oxidecomputer/dendrite#405.
rcgoodfellow
approved these changes
Nov 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously, SoftNPU didn't interact with multicast.
mc_group_createandmc_port_addreturned a stubbedOperationUnsupported, and port remove/destroy were no-ops.With the merge of bitmap-based replication in oxidecomputer/p4rs#240, SoftNPU can now replicate off of port bitmaps carried in the replication table entry itself. On Tofino, membership subscriptions live in the PRE and change independently of match-action state. On SoftNPU, membership is captured at write time, and a membership mutation involves rewriting these affected replication entries.
Includes:
egress.asic_id_to_portmaps ASIC port IDs to logical port numbers for indexing the decap bitmap (same as on Tofino).TableRemoveis sent beforeTableAddbecause p4rs drops an add whose key already exists and keeps the older entry (action and params).mcast::resetdeletes each known group's entries before the table clears. SoftNPU'sTable::clearis still a stub.del_entries_whereskips the stale-entry table sweep when the backend cannot list entries (e.g., in SoftNPU).