Skip to content

feat(sensitive): Paperless-ngx behind Caddy, rootless, with a CPU ceiling (#133) - #430

Merged
Gerrrt merged 5 commits into
mainfrom
gerrrt/paperless-ngx-sensitive-tier-ed6912
Sep 10, 2026
Merged

Gerrrt merged 5 commits into
mainfrom
gerrrt/paperless-ngx-sensitive-tier-ed6912

Conversation

@Gerrrt

@Gerrrt Gerrrt commented Sep 9, 2026

Copy link
Copy Markdown
Owner

What changed

Paperless-ngx lands in stacks/sensitive/ as the first household service behind Caddy: three containers with expose: and no ports:, routed as https://paperless.matrix.elysium. Nothing runs on the running lab — the tier's host is trinity, which #404 has not built; this is authored ahead of the hardware the way the foundation (#415) and stacks/lab were.

The stack

  • paperless — ghcr.io/paperless-ngx/paperless-ngx, rootless in upstream's own form (user: set, no USERMAP_*) as ${RENDER_UID} with cap_drop: [ALL]. The uid is the operator's on purpose: ./consume is a bind mount the operator drops scans into over SSH, and the consumer has to be able to delete what it consumed. cpus: 4 — the estate's first CPU ceiling, per the issue's "OCR will use everything it is given" — and 3072m, both stated as unmeasured on the hardware they are for. init: false, because s6-overlay refuses to be anything but PID 1 (booted with the default it exits with s6-overlay-suexec: fatal: can only run as pid 1). Not read_only, because OCR scratch under /tmp scales with the document and a tmpfs there is charged to the memory limit.
  • paperless-db — postgres:18, the major upstream pins; the volume mounts /var/lib/postgresql (the 18 layout), runs as 999 from the start, read_only with two tmpfs.
  • paperless-broker — valkey/valkey, the broker upstream moved to; init: false because the image's entrypoint is tini.
  • Caddyfile: one @paperless host block, with the proxy headers Django needs set on the compose side (PAPERLESS_USE_X_FORWARD_HOST, PAPERLESS_PROXY_SSL_HEADER).

What else changed, each because something required it

  • scripts/render-config.sh: three keys in COMPOSE_VARS, and a new block that creates gitignored bind-mount directories (consume/, export/) owned by the operator — Docker would create them as root and a rootless consumer could then delete nothing. A tracked mount that is missing dies instead of being papered over. Both paths exercised.
  • scripts/backup-volumes.sh: sentinels for the four Paperless volumes, read off the volumes after a boot and one consumed document — and for the foundation's three, which had none, so STACK=sensitive make backup refused on caddy-data before this. Caddy's were read off a started container; step-ca's is by construction (the entrypoint refuses to start without config/ca.json).
  • secrets/sensitive.example.yaml (three keys, with rotation notes), .env.example (URL, admin user, timezone, OCR language, worker/thread counts), seed-validation-env.sh, .gitignore, the stack README, the architecture row, the README tree line, the roadmap's ADR-0008: the sensitive-tier services and two new firewall rules are untracked #102 paragraph.

Why

#133. The archive is on this tier by content — tax returns, passports, medical records — and ADR-0023 classes it durable: the index can be rebuilt, the originals cannot, so the backup has to cover the archive and the database, which is what the sentinel table is for.

Blast radius

None to the running lab. trinity does not exist; nothing here is deployed. The three images were booted from their pinned digests on the monitoring host as throwaway pltest-* containers under a scoped Alertmanager silence (created and deleted in the same session), labelled homelab.logs=off, and removed afterwards along with the images.

  • No change to network segmentation or firewall rules — reached from Hicks under the existing 50→99 pass for 443; the name needs a host override on morpheus (ADR-0010's seventh), which is a build-time step under Build the sensitive-tier host on the ProDesk (ADR-0034) #404, not a rule
  • No new port published to a VLAN that could not already reach the service — Caddy's 443 stays the tier's only ports:
  • No credential added outside secrets/*.sops.yaml — PAPERLESS_SECRET_KEY, PAPERLESS_DBPASS, PAPERLESS_ADMIN_PASSWORD

Verification

  • make validate passes (worktree-only skips: the estate's timers)
  • Deployed to the lab and confirmed working — not applicable: the host is Build the sensitive-tier host on the ProDesk (ADR-0034) #404. What was done instead: the three containers booted rootless from the pinned images on the monitoring host; migrations and the index rebuild ran; CapEff/CapBnd 0 and NoNewPrivs 1 measured; a generated 200 dpi one-page scan dropped into consume/ was OCR'd to PDF/A-2b in 14.5 s and deleted as the operator's uid; document_exporter wrote a manifest into export/ owned by the operator; the broker's dump.rdb survives the SIGTERM a quiesce sends; working set 747 MiB idle, 825 MiB during OCR, 22 processes. check_compose_health.py --probe execs curl, pg_isready and valkey-cli in the three images; check_caddyfile.sh validates and formats; check_docs.py, check_image_pins.py, check_sops_rules.py and lint.sh pass; STACK=sensitive backup-volumes.sh --inventory lists all seven volumes.
  • Docs updated (stacks/sensitive/README.md, docs/architecture.md, docs/roadmap.md, README.md)

What make validate does not prove, in the README: that it runs on trinity; that the CA tree exists; that the name resolves or the leaf carries it as a SAN (make certs … --dns paperless.matrix.elysium); that 4 cores and 3 GiB fit OCR on the ProDesk. Still open before real data (ADR-0023): the off-estate encrypted copy, and a backup timer for this stack — the homelab-backup-volumes unit carries STACK=observability.

Refs #133, #404, #102.

🤖 Generated with Claude Code

Gerrrt and others added 5 commits September 9, 2026 23:24
…ling (#133)

The first household service in stacks/sensitive: Paperless-ngx with a
Postgres and a Valkey of its own, each with `expose:` and no `ports:`, routed
as https://paperless.matrix.elysium. Authored ahead of trinity (#404) the way
the foundation was, and booted from the pinned images on the monitoring host
before the file was written.

- paperless runs as ${RENDER_UID} with cap_drop [ALL] — upstream's rootless
  form — so a scan the operator drops into ./consume over SSH is one the
  consumer can delete. cpus: 4 is the estate's first CPU ceiling; 3072m is
  stated as unmeasured on the hardware. init: false, because s6-overlay
  refuses any PID but 1; not read_only, because OCR scratch charged to the
  memory limit would turn a long scan into an OOM kill.
- paperless-db is postgres 18 on the /var/lib/postgresql layout, as 999,
  read_only; paperless-broker is valkey with its own tini, init: false.
- render-config.sh creates the gitignored ./consume and ./export owned by
  the operator (Docker would create them root-owned), and dies on a tracked
  mount the checkout lacks. Three keys join COMPOSE_VARS and the seed.
- backup-volumes.sh gains sentinels for the four Paperless volumes, read off
  the volumes after a boot and a consumed document, and for the foundation's
  three, which had none — STACK=sensitive make backup refused before this.
- Caddyfile route, .env.example tunables, secrets template, stack README,
  architecture row, roadmap paragraph.

Refs #133, #404, #102.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Home Assistant (#427, #134) landed in stacks/sensitive under this branch.
Resolved on main's versions: the compose header counts two household
services and eight DIFFERENCEs (Paperless-ngx's CPU ceiling and writable
root become 7 and 8; its `init: false` lines point at Home Assistant's
DIFFERENCE 5), the Paperless block sits after Home Assistant's, Caddy's
`make certs` line carries both names, the Caddyfile keeps main's routing
preamble with the Paperless block above the catch-all, the secrets template
names four keys and Home Assistant's none, and the README merges both
services' notes. backup-volumes.sh also gains a sentinel for
home-assistant-config — ./.HA_VERSION, read off a boot of the pinned image
— because the merged stack's backup would otherwise refuse on that volume.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
#427 (Home Assistant, the Hue-bridge rule) and #431 (SNMPv3 per poll) each
added an ADR-0035 on separate branches and both merged on 2026-09-09, #431
second. check_docs.py's numbering assertion now fails on main and on every
PR that merges it. The one that landed second moves, as the check says:
the file, its H1, every link and every bare mention that is about SNMPv3.
The Hue-bridge references stay at 0035. README's ADR count goes to 36.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 06d0e77)
AdGuard Home (#429, #135) and the ADR renumbering (#452) landed under this
branch. Resolved on main's versions again: Paperless-ngx is the third
household service in the compose header, its CPU ceiling and writable root
are DIFFERENCEs 8 and 9 of nine, its block follows AdGuard's, Caddy's `make
certs` line and the README's cert and host-override notes carry all four
names, the secrets template counts five keys, and the stack README's table,
layout and validate sections merge all three services' notes.
backup-volumes.sh gains a sentinel for adguard-work — ./data/sessions.db,
read off a boot of the pinned image with its ports unpublished — so the
merged stack's backup does not refuse on that volume.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Immich (#435, #132) landed under this branch. Resolved on main's versions:
Paperless-ngx is the fourth household service in the compose header, its
CPU ceiling and writable root are DIFFERENCEs 10 and 11 of eleven, its
block follows Immich's, Caddy's `make certs` line and the README's cert and
host-override notes carry all five names, the secrets template counts six
keys, .env.example carries both new sections, and the stack README's
table, layout, backup and validate sections merge all four services' notes
and point the recipient half of the backup problem at #428.
backup-volumes.sh gains sentinels for immich-db (./PG_VERSION) and
immich-model-cache (./huggingface, present after the first model fetch),
each read off a boot of the pinned image, so every volume in the merged
stack has one — #428's first half.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Gerrrt
Gerrrt merged commit 9c59a62 into main Sep 10, 2026
3 checks passed
@Gerrrt
Gerrrt deleted the gerrrt/paperless-ngx-sensitive-tier-ed6912 branch September 10, 2026 05:58
@Gerrrt Gerrrt mentioned this pull request Sep 19, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant