Skip to content

The sensitive tier's secrets and volume backups open with trinity's key alone, and nothing alerts on a one-recipient stack #835

Closed

Description

@Gerrrt

What is true

The sensitive tier has held real data since 2026-10-02 (build-the-sensitive-tier-host.md:642: Vaultwarden and Paperless). Its secrets and its volume backups open with one key, trinity's.

  • .sops.yaml:89-91: the secrets/sensitive rule lists a single recipient, age1cpdxl…. :82-83 says so directly: "The technical second's recipient (ADR-0024, Add a second age recipient so decryption does not depend on one person #294) is still owed here; the tier has held real data since that day."
  • grep -c recipient: secrets/sensitive.sops.yaml returns 1.
  • scripts/backup-volumes.sh:258-259,1564-1567 encrypts the volume sets to the same recipient list. So vaultwarden-data, paperless-db-data and the rest open only with trinity's key.
  • docs/runbooks/restore-the-sensitive-tier.md:93: the second recipient "has to be on the sensitive rule before the first real item goes in."

#294 ("Add a second age recipient…") is closed. It landed before the sensitive rule existed (2026-09-28, #688), so the new rule was born with one key.

The Immich and Paperless library sets are partly covered, because household.recipients adds the technical second. The volume sets and the secrets file are not.

Why it matters

If trinity's disk dies, Vaultwarden's database and every backup of it die with the only key that opens them. That is the failure ADR-0024 was written to prevent.

soc.sops.yaml and lab.sops.yaml also have one recipient each. Those stakes are lower, and the lab is #671.

What closes it

  • make secrets-add-recipient STACK=sensitive PUBKEY=<technical second's age1…>, run on trinity. updatekeys re-wraps the data key; nothing is re-encrypted by hand.
  • One make backup STACK=sensitive, so the newest volume sets carry both recipients.
  • make secrets-verify-backup proves the second key opens both the secrets file and the newest volume set.
  • Decide whether soc gets the same treatment now or under The lab secrets are encrypted to one key, and it lives only on alexander #671's approach.
  • A guard so this cannot recur silently. SecretsKeyBackupUnproven (backup.rules.yaml:182) checks only the recipients that already exist. Add a check, in CI (scripts/check_sops_rules.py) or as a gauge plus rule, that fails or fires when a stack marked as holding real data has fewer than two recipients on its rule.

Done when

  • The secrets file and the newest volume set both decrypt with the technical second's key on a machine that is not trinity.
  • The guard exists and fails once, on purpose, against a one-recipient fixture.

Refs #294, #688, #671. Found in the 2026-10-03 repository review.

Activity

  1. added this to the trinity milestone on Oct 3, 2026
  2. added
    bugSomething isn't working
    securityA credential, an exposure, or a control that is weaker than recorded
    sensitivestacks/sensitive on trinity (VLAN 99)
    on Oct 3, 2026
  3. Gerrrt commented on Oct 4, 2026

    @Gerrrt
    OwnerAuthor

    Done, on both counts in Done when.

    The guard. scripts/check_sops_rules.py (#866, merged in f154884) reads each committed file's own sops: recipients. It fails if they differ from the file's .sops.yaml rule, or if a data-guarding file (observability, sensitive, tofu) opens with fewer than two keys. Its first CI run failed against the real one-recipient secrets/sensitive.sops.yaml, as intended. Ten --self-test cases cover the verdicts and the repair advice. Review then found that the advice could recreate the drift on the shared catch-all rule, and that was fixed before merge.

    The re-key. Run on trinity on 2026-10-03 with make secrets-add-recipient STACK=sensitive PUBKEY=age19mkg…. .sops.yaml and the re-keyed file were committed together (6490d49, merged via #866). Only the data-key stanzas changed; no encrypted value did.

    Proven on a machine that is not trinity, with the technical second's key alone:

    • Secrets file: make secrets-verify-backup STACK=sensitive KEY=… passed.
    • Newest volume set (20261004T020639Z, the first taken after converge picked up fix(secrets): two keys on the sensitive tier, and CI refuses one (#835) #866): its MANIFEST records recipient age19mkg76v0…,age1cpdxl6p4…. Every *.tar.gz.age in it was decrypted with age -d -i <second key> and streamed whole through tar -tzf, all ok. No plaintext was written to disk.

    What stays true. Volume sets taken before 2026-10-04T02:06Z still open with trinity's key alone. The runbooks now say so.

    Found along the way: add-recipient.sh, remove-recipient.sh and bootstrap.sh printed their verify hint without STACK, so a key on the catch-all rule would be proved against observability. Fixed in #882 (e95e475).

    🤖 Posted by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingsecurityA credential, an exposure, or a control that is weaker than recordedsensitivestacks/sensitive on trinity (VLAN 99)

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions