Skip to content

feat(smart): key SMART series on the port a drive is cabled to, not its letter (#745) - #759

Merged
Gerrrt merged 1 commit into
mainfrom
gerrrt/smart-baseline-stable-key-745
Sep 30, 2026
Merged

Gerrrt merged 1 commit into
mainfrom
gerrrt/smart-baseline-stable-key-745

Conversation

@Gerrrt

@Gerrrt Gerrrt commented Sep 30, 2026 •

Copy link
Copy Markdown
Owner

Closes #745, taking the issue's option 2 (by-path). The decision is recorded as ADR-0066, which supersedes ADR-0046's device key.

Why

smaug's boot SSD was sdc, then sdb, on two boots on 2026-09-29, with no disk moved. SmartDriveBadSectors paged on its recorded 4 sectors each time, and again on 2026-09-30, because the baseline row named the old letter.

What changes

  • collect-smart-state.sh: every per-device series gains a slot label, the drive's /dev/disk/by-path name.
    • Where udev offers two spellings (ata-6 and ata-6.0), the shortest wins.
    • Behind a Smart Array it is …:cciss,N.
    • Over SSH, or with no udev, it is the device label.
  • render-smart-baselines.sh: rows are keyed on host + slot. smaug's row is pci-0000:00:17.0-ata-6.
  • SmartDriveBadSectors: joins on(host, slot). A series with no slot, from a collector older than this (oracle's installed copy), uses its device as the slot. So oracle's row still names /dev/sda, and the rollout order between hosts doesn't matter for oracle.
  • A second bug this fixes: SmartDriveBadSectorsGrowing subtracts each series from its value a week earlier, matched by labels, and the letter was one of those labels. A drive with 4 sectors landing on a letter that held 0 would have read as growth of 4. With slot in the label set, a moved letter is a new series with no week-old point.

Tests

  • host.test.yaml:
    • The S3520 moving from sdc to sdb against one row: quiet.
    • A row still naming a letter: pages. That's ADR-0046's loud failure, kept.
    • Growing across a letter move: quiet.
    • The existing oracle cases now carry the device fallback.
  • collect-smart-state.sh --self-test: the slot survives a letter move and rides the model-labelled series, and by_path_of picks the shorter spelling and ignores partitions. 28/28 pass locally.
  • check_docs.py passes.
  • promtool test rules could not run locally (no promtool or Docker on this Windows machine), so CI's Validate configs is its first run.

Needs before merge

  • Confirm smaug's slot. Confirmed 2026-09-30: pci-0000:00:17.0-ata-6 -> ../../sdb, with an ata-N.0 spelling beside every disk, so the shortest-spelling rule picks ata-6. Originally: pci-0000:00:17.0-ata-6 is inferred from the S3520 being on ata6 of the 00:17.0 AHCI. On smaug: ls -l /dev/disk/by-path/ | grep -v part.

Deploy order after merge

  1. smaug first: re-fetch collect-smart-state.sh into /mnt/erebor/apps/stack and run the cron command once. Its series then carry slot. The rule in force until step 2 still joins on device, so nothing pages in between.
  2. Then the monitoring host: git pull && sudo make smart-state, and reload the rules.

Doing step 2 first would page for smaug until step 1 runs. oracle needs nothing now. When its collector is next reinstalled, its row changes to the slot --print shows.

Known consequence

SmartDriveUnsafeShutdownsGrowing stayed quiet on 2026-09-29 only because the letter moved. With a stable key it will fire on smaug's planned reboots, because the S3520 counts a clean shutdown as unsafe (hardware.md). That's #574's problem, and this PR exposes it rather than causes it.

🤖 Generated with Claude Code

…ts letter (#745)

smaug's boot SSD was sdc, then sdb, on two boots on 2026-09-29 with no
disk moved, and SmartDriveBadSectors paged on its recorded four each time
because the baseline row named the old letter. It paged again on
2026-09-30.

The collector now puts `slot` on every per-device series: the drive's
/dev/disk/by-path name (shortest spelling, then sort order), with
`:cciss,N` behind a Smart Array, and the device label where there is no
by-path name. The baseline is keyed on host + slot, and the rule joins
on(host, slot). A series from a collector older than this has no slot,
so the rule gives it its device as the slot, and oracle's row still
names /dev/sda.

It also fixes a second letter bug: SmartDriveBadSectorsGrowing matched a
series to itself a week earlier by letter, so a drive with 4 sectors
landing on a letter that held 0 would have read as growth.

ADR-0066 records the decision, superseding ADR-0046's device key. Tests
cover a letter move against one row (quiet), a row still naming a letter
(loud), the Growing rule across a move (quiet), and by_path_of's choice
between spellings.

Closes #745

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Gerrrt
Gerrrt merged commit 631ed06 into main Sep 30, 2026
7 checks passed
@Gerrrt
Gerrrt deleted the gerrrt/smart-baseline-stable-key-745 branch September 30, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Key the SMART baseline on something a reboot cannot move: smaug's /dev/sdX changes across boots

1 participant