Repository navigation
Description
Found on 2026-10-10 while building garuda (#921), the first real clone of template 910 (tpl-kali-saruman). 902 (tpl-kali, for ifrit, #790) shares the same preseed and package set, so it should inherit this.
What happened
tofu apply made VMID 162 from 910 and booted it. On that first boot cloud-init never ran:
cloud-init statusreportednot started.- The hostname stayed
tpl-kali-saruman. - There was no
analystuser and noauthorized_keys, so SSH fromphoenixwas refused (Invalid user analyst).
The datasource was fine. ds-identify.log shows Found single datasource: NoCloud, got label=cidata and returning 0.
The generator, though, was cut off mid-script. /run/cloud-init/cloud-init-generator.log ends at:
checking for datasource
The script's next line logs ds-identify rc=…, which never appears. So cloud-init.target was never linked into multi-user.target: /run/systemd/generator.early/ did not exist, and the target stayed inactive (dead).
In the same boot's journal, a few microseconds apart (systemd 262, Kali cloud-init 26.2-1):
[5.608679] systemd[1]: Detected first boot.
[5.608733] systemd-sslh-generator: Configuration directory '/etc/sslh/' does not exist! No units generated.
[5.608748] (generato[283]: '/usr/lib/systemd/system-generators/systemd-sslh-generator' terminated by signal ABRT.
[5.608829] systemd[1]: Failed to preset all unit: ... is transient or generated
[5.609155] systemd[1]: Failed to preset all unit ...: Read-only file system
sslh is there because kali-linux-headless depends on it, and the preseed installs that through kali-linux-default.
qm reboot 162 fixed it. On the second boot, which is no longer systemd's "first boot", the generator logged ds-identify rc=0 and enabled via …/cloud-init.target. cloud-init then ran: hostname garuda, user analyst, and the key in place.
Intermittent, not every time
Both scripts/packer-smoke.sh 910 runs, one after each build, cloned 910 and passed on the clone's first boot (hostname smoke-910, SSH as smoke). So it is 1 failure in 3 first boots so far, which looks like a race between generators on first boot rather than a fixed misconfiguration. Which of these kills the cloud-init generator is not known yet:
- the sslh generator's
ABRT; - first-boot
preset-allon a still read-only root; - systemd 262's generator handling.
A side effect worth knowing
The first boot without cloud-init still let sshd generate host keys. The second boot's cloud-init replaced them, so a key pinned between the two boots goes stale. Pin a Kali clone's host key only after cloud-init status says done.
Options
Not decided:
- Remove the trigger from the template. Purge
sslhinpacker/kali.pkr.hcl's provisioner, or keep it and give it an/etc/sslh/so its generator exits cleanly. This only helps if sslh is the cause. - Stop depending on the generator. Statically enable
cloud-init.targetfrom the template, e.g. amulti-user.target.wantslink. Then a killed generator no longer decides whether cloud-init runs. - Detect it. Make
packer-smoke.shassertcloud-init statusisdone, not only the hostname. Add a runbook check after a Kali clone's first boot: if it is not done,qm rebootonce. - Reproduce first: clone 910 several times and count the failures, to tell a race from a one-off.
garuda itself is fine now, after the reboot. This is about the next clone.
Activity
docs(lab): garuda's as-run log, and what the build corrected (#921)
feat(lab): garuda, a Kali Purple analyst workstation with dotfiles-De…
docs(packer): #1127 as measured — the fix proved on 12 of 12 first boots
Gerrrt commentedon Oct 10, 2026
Fixed in template 910 on 2026-10-10 (#1157).
Cause. On a clone's first boot, cloud-init's generator was cut off before it enabled
cloud-init.target. This affected 2 of 7 first boots of 910:garuda, and one of four disposable reproduction clones. Ongaruda's first boot, sslh's generator aborted alongside it.Fix (both in
packer/kali.pkr.hcl's shared provisioner):cloud-init.targetis now linked intomulti-user.target, so cloud-init no longer depends on the generator./dev/nulllink. Thesslhpackage stays installed.This is option 2 from the issue, plus option 1's masking. 902 inherits it when
ifritbuilds it.Proved. I rebuilt 910 with the fix. Then 12 of 12 first boots of disposable clones ran cloud-init: their own hostname,
status: done, and the generator completing. On the last 4, sslh's generator no longer aborts. An earlier version of my check counted an sslh abort on every boot because it matched its own command in the journal. Those sslh counts are void; the cloud-init counts stand.Not added: a
cloud-init statusassertion inpacker-smoke.sh. Its hostname and SSH checks already fail when cloud-init does not run.docs(packer): #1127's two changes are one fix, proved together
docs(packer): say which half of #1127's fix guarantees cloud-init starts
fix(packer): Kali clones no longer come up without cloud-init (#1160)
fix(packer): / last and no swap partition, so a bigger clone's root g…