Add internal reverse proxy for gateways (Caddy, wildcard via DNS-01)

cnx.router.proxy serves <service>.<site><n>.cnx.network with a real Let's
Encrypt wildcard obtained via a gateway-scoped TSIG key against ns1; Blocky
resolves the names to the router's LAN address, so they exist only
internally. First user: Omada UI on gw-cnx-1 (omada.cnx1.cnx.network).

docs/src/gateways.md

@@ -20,6 +20,7 @@ Naming: `gw-<city>-<n>`, e.g. `gw-cnx-1`.
 | IPv6         | DHCPv6-PD on ppp0, /64 per VLAN via SLAAC                                                                                                                                                     |
 | Bans         | CrowdSec + nftables bouncer (sshd log parsing)                                                                                                                                                |
 | Omada        | Optional per site: TP-Link Omada controller as a podman container                                                                                                                             |
+| Proxy        | Optional: Caddy reverse proxy for internal services under `*.<site><n>.cnx.network` with a real Let's Encrypt wildcard (DNS-01 against ns1)                                                   |
 | Management   | ZeroTier mesh: SSH, node_exporter, journald upload — like the fleet                                                                                                                           |
 | Storage      | Single-disk ZFS (zstd, `/var` its own dataset for snapshots)                                                                                                                                  |
 
@@ -57,17 +58,24 @@ Trust model: mgmt → everything; other VLANs → router DNS/DHCP + internet onl
    (dynamic PPPoE IP; clan connects over the mesh).
 3. Add its node_exporter and Blocky scrape targets in
    `modules/monitoring/server.nix`.
-4. `git add` the new machine directory — flake evaluation only sees
+4. If the site uses the internal proxy (`cnx.router.proxy.enable`), wire ns1:
+   import `(import ../../modules/dns/acme-gw-secret.nix "gw-<city>-<n>")` in
+   **both** the gateway's and ns1's configuration, and on ns1 add the
+   `dns-acme-gw-<city>-<n>-knot` generator, its `keyFiles` entry, and an
+   `acl_acme_gw_<city>_<n>` scoped to `_acme-challenge.<city><n>` on the
+   `cnx.network` zone (copy the `gw-cnx-1` blocks). Then
+   `clan vars generate ns1` and redeploy ns1.
+5. `git add` the new machine directory — flake evaluation only sees
    git-tracked files, so an untracked `machines/gw-…/` is silently ignored.
-5. `clan vars generate gw-<city>-<n>` — prompts for the site's PPPoE
+6. `clan vars generate gw-<city>-<n>` — prompts for the site's PPPoE
    credentials, mints the ZeroTier identity etc. (`nix flake check` fails
    until this has run, because mesh-hosts reads the ZeroTier IP var.)
-6. Boot the box from a NixOS installer USB on the local network, then:
+7. Boot the box from a NixOS installer USB on the local network, then:
    `clan machines install gw-<city>-<n> --target-host root@<lan-ip>`
-7. Check `facter.json` for the real NIC names, fix `wan.interface` /
+8. Check `facter.json` for the real NIC names, fix `wan.interface` /
    `trunkPorts` if the enumeration differs, and
    `clan machines update gw-<city>-<n>` (rides the mesh from then on).
-8. Add a row to the site table above and to the machines table in
+9. Add a row to the site table above and to the machines table in
    [Overview](./overview.md).
 
 ## Omada controller
@@ -80,6 +88,33 @@ gateway's mesh address from an admin machine. Controller state is under
 `/var/lib/omada`, declared as clan state (`clan.core.state.omada`); wiring it
 into the borgbackup instance is still a follow-up.
 
+## Internal reverse proxy
+
+`cnx.router.proxy.enable` puts Caddy on the gateway, terminating TLS for
+`<service>.<site><n>.cnx.network` (e.g. `https://omada.cnx1.cnx.network`) and
+forwarding to internal backends:
+
+```nix
+cnx.router.proxy = {
+  enable = true;
+  services.omada = {
+    backend = "https://127.0.0.1:8043";
+    insecureSkipVerify = true; # Omada's cert is self-signed
+  };
+};
+```
+
+- **Cert**: one real Let's Encrypt wildcard `*.<site><n>.cnx.network` per
+  gateway, issued via ACME DNS-01 (lego/RFC2136) against ns1 — works behind
+  PPPoE with zero inbound reachability, and browsers trust it without a CA
+  install. Each gateway has its own TSIG key (`acme_gw_<city>_<n>`) that ns1
+  scopes to `_acme-challenge.<site><n>` TXT records only.
+- **Resolution**: the names exist only internally — Blocky answers
+  `*.<site><n>.cnx.network` with the router's `lan` address; the public
+  `cnx.network` zone never carries them.
+- **Access**: `proxy.allowVlans` (default `mgmt` + `lan`) get 443 (and 80 for
+  the HTTP→HTTPS redirect). Not exposed to WAN, guest VLANs, or the mesh.
+
 ## Runbook
 
 - **PPPoE down**: `systemctl status pppd-wan`, `journalctl -u pppd-wan` on the