Repository navigation
feat(switch): poll each CRS326 port's PVID and alert on a change (#1387) - #1396
Merged
Merged
Conversation
Walk Q-BRIDGE-MIB dot1qPvid and BRIDGE-MIB dot1dBasePortIfIndex in the crs326 module, chained to ifDescr so each port's untagged VLAN carries the same port name as the IF-MIB series. Measured first on neo over v3: 24 bridge ports, bridge port N is ifIndex N+1, and the values match the port map. dot1qVlanStaticName is not implemented, so it is not polled. SwitchPortVlanChanged fires for an hour after any port's PVID changes. snmp-mibs.sh fetches the seven MIBs the two columns need. The regeneration also picks up the v0.31.0 generator's enum-label change for three unread iLO cpqSm2 index labels. Refs #1387. Refs #482. Refs #481. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are reported for the SNMP VLAN monitoring change, though deployment and live Prometheus verification remain pending.
No open issues 🎉
What changed in this PR
Adds SNMP-based per-port VLAN monitoring for the CRS326, enabling the port-map drift check to replace its retired MokerLink HTTP reader. Deployment and live Prometheus verification remain pending.
Changes:
- Polls port PVIDs and bridge-to-interface mappings, with chained lookups providing
ifDescrlabels. - Adds
SwitchPortVlanChangedand tests for changed, steady, and cleared states. - Fetches required MIBs, regenerates exporter configuration—including iLO enum-label updates—and updates documentation and rule counts.
| File | Description |
|---|---|
stacks/observability/snmp-exporter/snmp.yaml |
Adds generated bridge metrics and updates iLO enum index labels. |
stacks/observability/snmp-exporter/generator.yaml |
Adds PVID polling and chained interface-name lookups. |
stacks/observability/README.md |
Updates the Prometheus alert count. |
stacks/observability/prometheus/tests/network.test.yaml |
Tests VLAN-change alert firing, silence, and clearing. |
stacks/observability/prometheus/rules/network.rules.yaml |
Adds the one-hour VLAN-change warning. |
scripts/snmp-mibs.sh |
Fetches seven required bridge MIB dependencies. |
README.md |
Updates overall and Prometheus alert counts. |
docs/security.md |
Updates the total alert count. |
docs/runbooks/deploy-stack.md |
Updates expected loaded-rule counts. |
docs/observability.md |
Documents PVID monitoring, the alert, and revised coverage counts. |
docs/changelog.md |
Records VLAN monitoring and regenerated iLO label changes. |
Review effort: Balanced
Gerrrt
enabled auto-merge (squash)
November 11, 2026 03:48
# Conflicts: # docs/changelog.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
The
crs326SNMP module now walks Q-BRIDGE-MIB'sdot1qPvidand BRIDGE-MIB'sdot1dBasePortIfIndex. Each port's untagged VLAN lands in Prometheus with the sameifDescr="etherN"label as the IF-MIB series. A new warning,SwitchPortVlanChanged, fires for an hour after any port's PVID changes, as the prompt to record the change in the port map.scripts/snmp-mibs.shfetches the seven MIBs the generator needs for those two columns. Three come from net-snmp v5.9. The other four (Q-BRIDGE, P-BRIDGE, RMON2, TOKEN-RING-RMON) are not in net-snmp, so they come from the observium commit already pinned for UPS-MIB. Each one was removed in turn and the generator then failed with "Missing MIB".snmp.yamlis regenerated. Besides thecrs326additions, it picks up the v0.31.0 generator's enum-label change (feat: map enum lookup columns to string labels prometheus/snmp_exporter#1624), which feat(switch): poll the CRS326 over SNMPv3 and record it as built (#444, #84) #1303's regeneration did not include. Three iLO index labels (cpqSm2CommPort,cpqSm2NicLocation,cpqSm2NicStatsLocation) now carry the enum name instead of the number, across 49 metrics. Nothing in rules, dashboards or docs reads them. The same diff comes out ofmain's owngenerator.yamlwith the current MIBs.Why
Lemmiwinks' drift check read the switch's PVIDs from the MokerLink's
vlan.cgipage, and that page went with the swap (#444, Lemmiwinks#316). #482 asked for this over SNMP and was closed because the MokerLink had no Q-BRIDGE-MIB. The CRS326 has one. With the series in Prometheus, the port-map claim can come back as a query, with no switch login and no credential on oracle. The alert catches an unrecorded port move like #481.The issue's two open questions were answered by walking the live switch over v3 on 2026-11-11 with
scripts/snmp-walk.sh:dot1qPvidis the bridge port'spvidunder bridge VLAN filtering. It returns 24 rows, one per bridge port (ether1–24; the SFP+ cages are not bridge ports), and every value matchesnetwork.md: port 1 is 7,ether17/smaug is 40,ether20is 30.ifIndex = port + 1carries over:dot1dBasePortIfIndexmaps bridge port N to ifIndex N+1 on every row.dot1qVlanStaticNameis not implemented (0 rows), so it is not polled.Refs #1387. Refs #482. Refs #481.
Blast radius
prometheus's snmp-exporter and the switch's scrape. If the new columns broke the walk,neo's scrape would fail andSnmpTargetUnreachablewould say so, andSwitchInterfaceDowncoverage would go with it. The walk was measured first: 1 GETBULK each, 0.55 s and 0.08 s, and every overshoot varbind (an INTEGER and an OID) decodes. The iLO label change touches onlycpqSm2comm-port and NIC series that nothing reads.secrets/*.sops.yamlVerification
scripts/snmp-walk.sh --device neoagainst the live switch, for the three OIDs above (results in Why).make snmp-mibs && make snmp-generate: metrics went from 1792 to 1794, and the 7 placeholders survived. A trial generator run in scratch, with each MIB removed in turn, established the minimal set.promtool test rules network.test.yaml: SUCCESS. The new case fires at 31m on a 50→99 change, stays quiet on a steady port, and clears at 95m../scripts/lint.sh: passes.python3 scripts/check_docs.py:docs OK — 161 Prometheus + 21 Loki rules.make validate: all checks passed, 5 skipped (not run): the offsite path refusal (/tmpis tmpfs here) and four unit/timer checks that only run in the deployment checkout. CI runs them.Not yet done: deploy. After merge,
make renderfrom the main checkout, a force-recreate ofsnmp-exporter, then confirm thatdot1qPvid{instance="10.7.7.2"}returns 24 series withifDescrand thatSwitchPortVlanChangedis loaded and inactive. Poll the CRS326's VLAN table: dot1qPvid now exists, and the drift check lost its port-map read with the MokerLink #1387 is then closed by hand with that proof. Lemmiwinks#316 can then querydot1qPvidinstead of retiring the row.make validatepassesDeployed to the lab and confirmed working
Docs updated (
docs/, service README, ordocs/roadmap.md)🤖 Generated with Claude Code