Skip to content

A stale .part on the offsite medium accumulates, and one shape of it blocks every future copy #613

Closed

Description

@Gerrrt

A .part from an older stamp is never cleaned, and one shape of it blocks every future copy

copy_set removes only the leftover belonging to its own stamp
(scripts/backup-offsite.sh:520-522):

# A .part of this exact stamp is this script's own leftover from a copy that
# died; nothing else writes that name.
rm -rf -- "${part}"

Retention will not touch anything else: prune_kind classes an unrecognised directory as a
stray, counts it and explicitly leaves it
(:552-556), and prune_exports only removes
names matching is_export plus their sidecar, so config-….sops.yaml.part is immortal too.

Two consequences

Dead bytes accumulate on a medium sized to the floor. ADR-0048 puts the requirement at
"a few gigabytes free is the floor" because a volume set is ~1.7 GB. A failed volume copy
leaves up to 1.7 GB that the pre-flight room check
(:678-683) then counts against the next copy,
which refuses rather than fills. The medium gets smaller every time a copy dies.

One crash window wedges the medium outright. If the copy died between cp MANIFEST and
the mv (:530-532), the stale <stamp>.part/
has a MANIFEST. Then:

  1. sets_in finds it — find -mindepth 2 -maxdepth 2 -name MANIFEST
    (:429) does not care about the .part suffix.
  2. verify_set_dir rejects the name — is_stamp fails, so
    red "refusing to check a set with an unexpected name" (:444).
  3. verify_medium returns 1, and the copy path dies before writing anything
    (:624-630).

The medium is then un-copyable until a human deletes that directory by hand — discovered on
a ninety-day visit, by someone who may not be the person who wrote the script.

The runbook has no row for it

docs/runbooks/copy-the-backups-offsite.md:161-174
covers differs from its MANIFEST entry and differs from its recorded sha256, but not
refusing to check a set with an unexpected name. The error names itself but the table does
not say what to do about it.

Suggested shape

  • Sweep *.part under the kind directories at the start of a copy run — they are only ever
    this script's own leftovers, and the comment at :520 already says so. Age them if that
    feels too eager, but an unbounded .part on a size-constrained medium has no defenders.
  • Or exclude *.part from sets_in so a half-written directory cannot present as a set,
    which fixes the wedge without touching retention.
  • Either way, add the missing runbook row with the by-hand repair.

Found while auditing ADR-0048's merged mechanism for #573; not touched by #610, which was
documentation only.

Refs #573, #588, ADR-0048.

Activity

  1. added
    bugSomething isn't working
    docsImprovements or additions to documentation
    on Sep 22, 2026
  2. added a commit that references this issue on Sep 23, 2026
  3. added a commit that references this issue on Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocsImprovements or additions to documentation

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions