Files

T

Berwn 044891927b Back up Knot DNSSEC keystore from ns1 to control via borgbackup

clan borgbackup instance: control serves repos, ns1 backs up its
clan.core.state (the KASP keystore at /var/lib/knot) nightly over the
mesh with repokey encryption. ns1 maps the control machine name to its
ZeroTier address so the borg@control repo resolves.

Run `clan vars generate ns1` before deploy to mint the borg keypair.

2026-06-17 15:06:58 +07:00

2.3 KiB

Raw Blame History

Infra roadmap

Prioritized backlog for the cnx-network clan. See docs/ for how the current pieces work.

1. Alerting (done — pending deploy)

Rules evaluated by vmalert against VictoriaMetrics on control, declared in modules/monitoring/alerts.nix:

SOA serial divergence between ns1 and ns2 (secondary out of sync)
Zone-expiry countdown on the secondary approaching zero (transfers failing)
Any scrape target down (up == 0)
Root filesystem nearly full

Delivery stays minimal for now (notifier.blackhole): vmalert remote-writes alert state back to VM, so firing alerts show up as the ALERTS series in Grafana. Wiring a real notifier (Matrix) is a later step — drop blackhole and set settings."notifier.url" to an Alertmanager.

2. Backups of critical state (DNSSEC done — pending vars + deploy)

clan borgbackup instance in clan.nix: control is the server (repos under /var/lib/borgbackup/<client>), ns1 the client. ns1 declares clan.core.state.knot.folders = [ "/var/lib/knot" ], so the Knot KASP keystore is backed up nightly (01:00) over the mesh with repokey encryption — control never holds plaintext. ns1 maps the control machine name to its mesh IP via networking.hosts so the borg@control repo resolves.

Before deploy: clan vars generate ns1 (YubiKey) to mint the borgbackup ssh keypair + repokey; control won't evaluate until ns1's public key exists. Then deploy ns1 and control.

DNSSEC key material on ns1 (KSK/ZSK in Knot's KASP store) — losing it forces an emergency DS rollover at the registrar
VictoriaMetrics TSDB on control (optional, retention is 180d) — deferred; regenerable over time and control is the backup server, so this needs a second client→server pair (e.g. control→ns2) rather than the same topology

3. Blackbox DNS probing

blackbox_exporter on control doing real DNS + DNSSEC-validation queries against ns1/ns2 — catches outside-in resolution failures the Knot stats miss

4. Third secondary off Hetzner (resilience)

A secondary nameserver on a different provider/network so a single-provider outage doesn't take all authoritative DNS down (architectural — new machine)

5. Centralized logs

VictoriaLogs on control to grep journald across all three hosts, pairing with the existing VictoriaMetrics setup

2.3 KiB Raw Blame History