Repository navigation
Description
As decided by ADR-0036, which the body below predates
Both premises as filed are wrong, and were wrong on the day they were written. The body says pfSense, the APC NMC and the iLO support SNMPv3 authPriv and "the MokerLink switch does not, which is the blocker." ADR-0036 measured the opposite on both counts: morpheus is the device that cannot — bsnmpd is the only daemon that serves the pf MIB and pfSense writes no USM user for it — and neo answers v3 on the wire and was never the blocker. v3 is decided per poll, by where the poll travels: the iLO and the UPS card go to authPriv; the firewall stays on v2c over bsnmpd; the MokerLink stays on v2c until #444 replaces it, and #84 rides along.
What is done
The repository half, in #431: the generator's auth blocks exist for each device, the secrets stanzas are shaped, rotate-snmp-community.md §4 is the device-side procedure. Nothing device-side has been run — all four auth blocks in stacks/observability/snmp-exporter/generator.yaml still read version: 2 (auth_apc, auth_ilo, auth_mokerlink, auth_pfsense).
What remains
-
rotate-snmp-community.md§4 againstshiva(the iLO) — physical access; the iLO can drop the session on save - the same against
mjolnir(the UPS card) -
auth_iloandauth_apctoversion: 3in the generator, secrets rendered,snmp-verify.shclean -
docs/roadmap.md's Security section stops bundling this with the switch
docs/roadmap.md lists this under Decided but not built, which is right.
Body as filed, superseded 2026-09-19
SNMPv2c sends the community in cleartext, so anyone with a port on the
management VLAN can read it off a single packet. Per-device communities limit
what one captured packet grants; they do not make the transport private.
pfSense, the APC NMC and iLO all support SNMPv3 authPriv. The MokerLink switch
does not, which is the blocker for doing it uniformly — and a fleet where
three devices are encrypted and one is not still leaks the fourth.
scripts/snmp-verify.sh:20 notes the same constraint.
Worth deciding: move the three that can and accept a mixed estate, or treat this
as blocked until the switch is replaced.
Tracked in docs/roadmap.md; filed as an issue so it has a place to be discussed and closed.
Activity
Gerrrt commentedon Sep 9, 2026
Decided in ADR-0035, PR #431. Neither of the two framings in the issue survived a check against the devices on 2026-09-09:
morpheusanswers from bsnmpd; pfSense's GUI writes no USM user for it, and bsnmpd is the only daemon that serves the pf MIB. The Net-SNMP package (would do SHA/AES v3) is installed on the box, disabled, and has no pf module — using it would tradePfNotRunning,PfStateTableNearLimitandPfMemoryDropsIncreasingfor encryption of the credential that reads those tables. Not worth it;morpheusstays v2c.10.7.7.2gets a USM Report ("Unknown user name"), exactly like the other three. Whether the web UI has a page to create a user is unchecked — its login is the only way to look. So ADR-0018's "cannot do SNMPv3 at all" is narrowed to "may have no way to create a user". Worth one look at the next login; recorded inhardware.mdeither way.10.0.30.10reads the community every minute. The other three polls ride on Winterfell, where a sniffer already sits on the segment with the firewall's UI and the monitoring host.So: per poll, mixed on purpose.
shivafirst, thenmjolnir. authPriv only —snmp-targets.sh --checkrefuses anything else.The repository half is in #431: a device's version and key names now come from its auth block in
generator.yaml,snmp-verify.sh/snmp-walk.shspeak v3 through the samesnmp.confroute, and USM rejections are reported as rejections rather than timeouts. Tested against the live iLO with a user that does not exist (net-snmp:Unknown user name; the pinned exporter:incoming packet is not authentic, discarding). All four devices are still v2c in that PR — device first, then repository.What is left is the device side, in
rotate-snmp-community.md§4: create the user on the iLO (physical access, it can drop the session on save), flipauth_ilo, two sops keys in place of one,make render && make reload, verify, switch SNMPv1 Request off, prove it with--old. Then the card. Please paste thesnmp-verifyoutput here after each one, including the--oldline, and thescrape_duration_secondsbefore and after — ADR-0035 says the v3 cost is measured, not assumed.Merge pull request #431 from Gerrrt/gerrrt/snmpv3-authpriv-migration-…
Merge origin/main into claude/homelab-almanac-review-uwswis
[-]Move to SNMPv3 authPriv where the hardware supports it[/-][+]Move the iLO and the UPS card to SNMPv3 authPriv (ADR-0036)[/+]feat(snmp): poll shiva over SNMPv3 authPriv (#85)
chore(secrets): the iLO's community becomes two SNMPv3 passphrases (#85)
Merge pull request #581 from Gerrrt/gerrrt/snmpv3-ilo-ups-authpriv-76…
Gerrrt commentedon Sep 20, 2026
shivais polled over SNMPv3 authPriv since 2026-09-20 22:28 UTC (#581, runbook §4.1–§4.4). The device went first: theprometheususer (SHA, AES) was created on the iLO with SNMPv1 Request still enabled, thenauth_iloflipped in the repository and the stack rendered and reloaded from the main checkout.snmp-verify.sh --device shivafrom the branch, before the exporter changed — thePASSwent over v3,--dry-runshowingshiva … v3 SNMP_AUTHPASS_ILO SNMP_PRIVPASS_ILO:After
make render && make reload, through the running exporter with the rendered v3 block: HTTP 200, 3866cpq*series,sysDescras above, and the exporter's own log clean ofincoming packet is not authenticsince the reload.up{job="snmp",instance="10.0.30.10"}was1on the first sample after the reload.Scrape cost, per ADR-0036 ("measured, not assumed") — the v3 discovery round trip is in the noise on a 12-second walk:
scrape_duration_secondsonshivaA one-hour average after the move follows once there is one.
One thing the dry run in #431 could not see, now fixed in #581: the generator writes its stock default into a v3 block that has no
community:line, and gitleaks' community rule correctly fails validate on the generatedsnmp.yaml. A v3 block now carries an explicit empty string,snmp-targets.sh --checkrequires it, and the three documented copies of the shape show it.Still open on this device — §4.5, by hand: SNMPv1 Request off on the iLO, then
./scripts/snmp-verify.sh --oldwith the communitySNMP_COMMUNITY_ILOheld, wantingrejected; then a second--device shivaverify. The docs claim the v3 poll and nothing about v1 until that output is here.mjolniris next, same procedure.Gerrrt commentedon Sep 20, 2026
§4.5 done on
shiva: SNMPv1 Request is off, and the iLO refuses its own former community.snmp-verify.sh --oldreturnedrejectedfor the device — the check that matters, because it probes over v2c whatever the device speaks now. The move is complete on this device.Confirming verify afterwards, since switching v1 off is another save of the iLO's SNMP page:
The v3 scrape cost, now measured over a full hour rather than from one walk, which is what ADR-0036 asked for:
scrape_duration_secondsonshiva60 samples in that hour, no gaps, and
min_over_time(up{job="snmp"}[30m])is1for all four devices — the discovery round trip per scrape session costs nothing measurable against a 12-second walk.So on this device the whole of §4 is done: the user exists, the exporter polls it over authPriv, v1 is off and proved off. The documents now say so rather than pointing here —
SECURITY.md,docs/security.md,docs/hardware.mdand the roadmap bullet.mjolniris what is left, and it closes this issue. Same four steps, device first: two passphrases, a user profile on the NMC with SHA and AES and an access-control entry for10.0.99.20, thenauth_apcto the v3 block, then §4.5 on the card's Configuration → Network → SNMPv1 → Access.docs(snmp): the card's move resolves what the UPS shutdown runbook wa…
Merge pull request #592 from Gerrrt/gerrrt/snmpv3-ilo-ups-authpriv-76…
Gerrrt commentedon Sep 21, 2026
mjolniris polled over SNMPv3 authPriv since 2026-09-21 (#592). The profile was created on the NMC on 2026-09-20 beside the iLO's —prometheus, SHA, AES, an access-control entry for10.0.99.20and nothing else — and the repository half landed today.snmp-verify.sh --device mjolnirfrom the branch, over v3, before the exporter switched:After
make render && make reload, through the running exporter with the rendered v3 block: HTTP 200, 58 series, 5 packets, 0.03 s, and the values that matter still reading —upsAdvTestDiagnosticSchedule8 (biweekly since last test),upsTestResultsSummary1 (donePass),upsEstimatedMinutesRemaining47. Noincoming packet is not authenticin the log.up{job="snmp"}is1for all four devices.The v3 cost here is nothing: 0.03 s against the 0.03 s v2c 24-hour average. The card's walk is 58 varbinds, so the one discovery round trip is the whole difference and it does not show at this resolution.
What moving it found.
shut-down-on-the-ups.mdwas written while the card was still on v2c and named the v3 keysSNMP_AUTHPASS_MJOLNIR/SNMP_PRIVPASS_MJOLNIR. Those names have never existed — every tool derives them from the auth label,auth_apc, not from the device — so anyone building the NUT server would have searched SOPS for a key that is not there. Corrected, along with the conditional that has now resolved. And §4.5 gained the check ADR-0049 asked it for: anything else holding the device's community moves before SNMPv1 goes off, because a consumer left on v2c is what keeps v2c enabled and it fails silently, the exporter staying green throughout. That consumer does not exist yet here — the NUT package is installed and unconfigured — so there was nothing to move.What is left, and it is the last thing on this issue: the card's Configuration → Network → SNMPv1 → Access → disabled, then
./scripts/snmp-verify.sh --oldwith the communitySNMP_COMMUNITY_APCheld, wantingrejected, then a second--device mjolnir.Merge pull request #597 from Gerrrt/gerrrt/snmpv3-close-85-a41c73
Gerrrt commentedon Sep 21, 2026
Closed. Both devices ADR-0036 names are on SNMPv3 authPriv, and neither answers a community.
mjolnir's SNMPv1 access went off in the card's web UI, andsnmp-verify.sh --oldreported its former communityrejected— the check that proves it, because that probe is v2c whatever the device speaks now. The confirming verify afterwards, since switching v1 off is another save of that page:The cost on the card, measured rather than assumed. The walk is 58 varbinds, so one engine-discovery round trip is the whole difference:
scrape_duration_secondsonmjolnirTwo transients, named rather than averaged away: the first v3 scrape took 0.188 s, and one scrape took 10.0 s at the moment SNMPv1 was switched off — the NMC restarting its network interface on save, exactly as the runbook's Before you start warns. The next scrape was 0.041 s,
upnever dropped, and the UPS series all still read (58 of them;upsTestResultsSummary1,upsAdvTestDiagnosticSchedule8,upsEstimatedChargeRemaining100). On the iLO the same measurement was 12.09 s against 12.05 s.The estate is mixed on purpose, and that was the whole point of the decision. The firewall stays on v2c because
bsnmpdis the only daemon serving the pf MIB; the switch stays until #444 replaces it, and ADR-0041 commissions the CRS326 with an authPriv user by this same runbook. Both of those polls ride on Winterfell.One thing this issue caused that is worth recording. While finishing it, a command meant to inspect a single auth block printed the whole rendered
snmp.yaml, whose redaction covered thepasswordfields and not thecommunityones — putting both live v2c communities into a session transcript. A value-level sweep found them in a third file too, from an unrelated project and older than today. All redacted in place at mode 600, nothing committed or off the host, and the four v3 passphrases checked individually and found nowhere. Not rotated, by decision: the MokerLink's value retires with the hardware within days, and the firewall's poll never leaves Winterfell. It is a row inSECURITY.mdnow, beside the 2026-08-20 one it resembles.Closed by #597; the two moves were #581 and #592, on the tooling #431 built.