Author SHA1 Message Date
kurogeek 596ac1f4bb router: make CrowdSec opt-in (crowdsec.enable, default off)
Not every site wants the ban engine (hub sync needs internet at
activation, and it is one more moving part on a small box). Gate
crowdsec.nix on a new `crowdsec.enable` option like `omada.enable`.

gw-cnx-1 keeps it on; the VM test drops its mkForce overrides.
2026-09-21 07:20:17 +00:00
kurogeek e1e18dd9f3 router/tests: cover the staging uplink as NATed fallback exit
Neither regression fixed by the previous commit was observable: the VM
test had no staging port, and pinging the ISP's PPPoE address only needs
ppp0's connected route, not the default route pppd refused to install.

Add a fifth node, `oldlan`: a networkd DHCP server on vlan 4 handing gw
its staging lease, with a second address (203.0.113.1) that gw can only
reach through that lease's default route. It has no route back to the
VLANs, so client pings only work if gw masquerades.

Proven: pppd's metric-0 default wins over the DHCP one while ppp0 is up;
lan reaches the old LAN via the staging port and iot (allowWan = false)
does not; after stopping pppd the staging route carries WAN traffic with
the same allowWan split; restarting pppd makes ppp0 preferred again.
2026-09-21 04:32:14 +00:00
kurogeek 8090ab3e6d router: let allowWan VLANs out through the staging uplink
With stagingPort set, the box itself had internet over the staging DHCP
uplink but LAN/Wi-Fi clients had none: forward and masquerade were scoped
to ppp0 only. Worse, pppd's `defaultroute` refuses to install its route
while the staging DHCP default route (metric 1024) exists ("not replacing
existing default route"), so even a live PPPoE session was never used.

- firewall: forward-allow + masquerade allowWan VLANs -> stagingPort in a
  separate `router-staging-nat` postrouting chain (networking.nat only
  takes one external interface). Same allowWan set as nixos-nat.
- pppoe: `defaultroute-metric 0`, so pppd only checks for a metric-0
  default route, installs ppp0 as the preferred exit and removes it on
  hangup, leaving the staging route as the fallback.
2026-09-21 04:32:07 +00:00
kurogeek 4146f2c878 router/dhcp: drop Restart= already set by the nixpkgs kea unit 2026-09-18 10:08:03 +00:00
kurogeek 9ea058bfb8 router/dhcp: make Kea wait for its VLAN sockets
kea-dhcp4-server orders after network-online.target, which under networkd
only waits for the WAN carrier: the vlan-* links are RequiredForOnline=no.
On stellio Kea started before networkd had addressed vlan-lan/vlan-mgmt,
failed both binds and, per its default, kept running with no socket at
all (ss -ulnp showed nothing on :67). Wireless and wired clients'
DISCOVERs reached vlan-lan and were never answered.

Set service-sockets-require-all with a bounded retry so Kea keeps trying
while networkd catches up, and Restart=on-failure on the unit for the
case it still gives up.

Verified with checks.x86_64-linux.router.
2026-09-18 10:06:01 +00:00
kurogeek f98226bc6e router/wifi: assert no supplicant/NetworkManager owns the radios
With the AP-typed match, the station-mode netdev is unmanaged until
hostapd flips it, so any wpa_supplicant/iwd/NetworkManager in the
importing config would grab the radio first and keep resetting it to
managed mode under hostapd. Fail at eval instead of at runtime.
2026-09-18 09:36:07 +00:00
kurogeek d4e98d8dd5 router/wifi: match BSS .network files on WLANInterfaceType=ap
networkd enslaves the radio to br0 once, at link init. On real hardware
(GL-MT6000, mt798x-wmac) that happens while the netdev is still in station
mode, so the kernel rejects the bridge join (IFF_DONT_BRIDGE), the link is
parked in networkd's failed state and never retried: link_reconfigure_impl()
is a no-op while the matching .network file is unchanged, and the udev
'iw set type __ap' hook meant to pre-empt this loses the race. Wireless
clients associate but their DHCP never reaches vlan-lan.

Match on WLANInterfaceType=ap as well as the name. The station-mode netdev
then matches nothing (unmanaged) and the file first matches when hostapd
has switched the radio to AP and raised carrier, so the enslave succeeds
on the first try. Drop the udev hook.

Verified with checks.x86_64-linux.router (hwsim STA lease over the bridge).
2026-09-18 09:22:57 +00:00
kurogeek 60aac6efb2 services/router: init 2026-09-17 08:57:32 +07:00
12 changed files with 250 additions and 107 deletions
+4 -4
View File
@@ -21,14 +21,14 @@ mgmt-only trust model, SSH exposure and the Wi-Fi bridge ports. Run it with
## What each gateway runs ## What each gateway runs
| Function | Implementation | | Function | Implementation |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| WAN | PPPoE (`pppd`), per-site ISP credentials via clan vars prompts; `wan.vlanId` when the ISP tags the session (AIS: 10); `wan.macAddress` to clone the old router's MAC if the ISP has it pinned | | WAN | PPPoE (`pppd`), per-site ISP credentials via clan vars prompts; `wan.vlanId` when the ISP tags the session (AIS: 10); `wan.macAddress` to clone the old router's MAC if the ISP has it pinned |
| LAN | VLAN-filtering bridge `br0` over the trunk ports (networkd); `accessPorts` pin a port untagged to one VLAN — convention: the last copper port is an untagged `mgmt` recovery port | | LAN | VLAN-filtering bridge `br0` over the trunk ports (networkd); `accessPorts` pin a port untagged to one VLAN — convention: the last copper port is an untagged `mgmt` recovery port; `stagingPort` turns a spare port into a DHCP uplink into the old LAN while the box runs alongside the router it replaces (drop at cutover) |
| Firewall/NAT | nftables: default-deny WAN, no inter-VLAN, MSS clamp, v4 NAT | | Firewall/NAT | nftables: default-deny WAN, no inter-VLAN, MSS clamp, v4 NAT |
| DHCP | Kea, one subnet per VLAN | | DHCP | Kea, one subnet per VLAN |
| DNS | Blocky (blocklist resolver), metrics on :4000 scraped by control | | DNS | Blocky (blocklist resolver), metrics on :4000 scraped by control |
| IPv6 | DHCPv6-PD on ppp0, /64 per VLAN via SLAAC | | IPv6 | DHCPv6-PD on ppp0, /64 per VLAN via SLAAC |
| Bans | CrowdSec + nftables bouncer (sshd log parsing) | | Bans | Optional per site (`crowdsec.enable`): CrowdSec + nftables bouncer (sshd log parsing) |
| Omada | Optional per site: TP-Link Omada controller as a podman container | | Omada | Optional per site: TP-Link Omada controller as a podman container |
| Wi-Fi | Optional: hostapd on the router's radios; each SSID (`wifi.networks`) is an untagged access port of its VLAN, passphrases via vars prompts — see `modules/clan/router/README.md` | | Wi-Fi | Optional: hostapd on the router's radios; each SSID (`wifi.networks`) is an untagged access port of its VLAN, passphrases via vars prompts — see `modules/clan/router/README.md` |
| Proxy | Optional: Caddy reverse proxy for internal services under `*.<site><n>.cnx.network` with a real Let's Encrypt wildcard (DNS-01 against ns1) | | Proxy | Optional: Caddy reverse proxy for internal services under `*.<site><n>.cnx.network` with a real Let's Encrypt wildcard (DNS-01 against ns1) |
@@ -71,7 +71,7 @@ Trust model: mgmt → everything; other VLANs → router DNS/DHCP + internet onl
installer: `ls -l /dev/disk/by-id/`). installer: `ls -l /dev/disk/by-id/`).
2. Add the machine to `inventory.machines` in `clan.nix`, to the `router` 2. Add the machine to `inventory.machines` in `clan.nix`, to the `router`
instance in `inventory.nix` (`roles.default.machines.gw-<city>-<n>.settings`: `site`, instance in `inventory.nix` (`roles.default.machines.gw-<city>-<n>.settings`: `site`,
`siteId` (next free number), port names, VLANs, `omada.enable`; keep the `siteId` (next free number), port names, VLANs, `omada.enable`, `crowdsec.enable`; keep the
`mgmt`/`lan` VLANs), and to the machine list in `modules/mesh-hosts.nix`. `mgmt`/`lan` VLANs), and to the machine list in `modules/mesh-hosts.nix`.
Do **not** add it to `modules/hosts.nix` (dynamic PPPoE IP; clan connects Do **not** add it to `modules/hosts.nix` (dynamic PPPoE IP; clan connects
over the mesh). over the mesh).
+7 -4
View File
@@ -40,10 +40,11 @@ in
wan.interface = "enp1s0"; wan.interface = "enp1s0";
wan.vlanId = null; # this ISP runs PPPoE untagged on the port wan.vlanId = null; # this ISP runs PPPoE untagged on the port
wan.macAddress = "a8:b8:e0:01:06:87"; wan.macAddress = "a8:b8:e0:01:06:87";
trunkPorts = [ trunkPorts = [ "enp2s0" ];
"enp2s0" # STAGING (remove at cutover, move enp3s0 back into trunkPorts):
# "enp3s0" # STAGING: serves as the uplink until cutover (see the machine config) # DHCP-client uplink into the existing OPNsense LAN so the box has
]; # internet + mesh while it runs alongside the old router.
stagingPort = "enp3s0";
# Dedicated on-site recovery port: untagged mgmt, always available even # Dedicated on-site recovery port: untagged mgmt, always available even
# if the switch config is broken. # if the switch config is broken.
accessPorts.enp4s0 = "mgmt"; accessPorts.enp4s0 = "mgmt";
@@ -76,6 +77,8 @@ in
}; };
# This site runs the Omada controller for its APs/switches. # This site runs the Omada controller for its APs/switches.
omada.enable = true; omada.enable = true;
# sshd ban engine (was unconditional before the option existed).
crowdsec.enable = true;
# Internal reverse proxy: real wildcard cert *.cnx1.cnx.network; Blocky # Internal reverse proxy: real wildcard cert *.cnx1.cnx.network; Blocky
# resolves the names to the router's LAN address for mgmt+lan clients. # resolves the names to the router's LAN address for mgmt+lan clients.
-12
View File
@@ -16,18 +16,6 @@
builtins.hashString "sha256" config.networking.hostName builtins.hashString "sha256" config.networking.hostName
); );
# STAGING (remove at cutover, and restore enp3s0 to trunkPorts in clan.nix):
# DHCP-client uplink into the existing OPNsense LAN so the box has internet +
# mesh while it runs alongside the old router. Default-deny firewall on this
# interface (it's in no VLAN zone); PPPoE simply retries until the WAN port
# is cabled. Do NOT connect the trunk ports to the production switch while
# staging — Kea on tag 10 would fight the OPNsense LAN DHCP in one broadcast
# domain.
systemd.network.networks."05-staging" = {
matchConfig.Name = "enp3s0";
networkConfig.DHCP = "ipv4";
};
time.timeZone = "Etc/GMT-7"; # UTC+7 (Thailand, fixed offset, no DST) time.timeZone = "Etc/GMT-7"; # UTC+7 (Thailand, fixed offset, no DST)
services.chrony.enable = true; services.chrony.enable = true;
} }
+6 -4
View File
@@ -3,10 +3,11 @@
Turns a machine with several NICs into a site gateway: PPPoE WAN (ISP Turns a machine with several NICs into a site gateway: PPPoE WAN (ISP
credentials via vars prompts), a VLAN-filtering bridge over the LAN ports with credentials via vars prompts), a VLAN-filtering bridge over the LAN ports with
one L3 interface per VLAN, Kea DHCP and Blocky DNS per VLAN, nftables one L3 interface per VLAN, Kea DHCP and Blocky DNS per VLAN, nftables
firewall/NAT, DHCPv6-PD, CrowdSec, an iperf3 server and a WAN speed-test firewall/NAT, DHCPv6-PD, an iperf3 server and a WAN speed-test timer.
timer. Optional: a Wi-Fi access point on the router's own radios (hostapd), Optional: a Wi-Fi access point on the router's own radios (hostapd), CrowdSec
the TP-Link Omada controller (podman) and an internal Caddy reverse proxy with the nftables bouncer (sshd log parsing), the TP-Link Omada controller
with a real wildcard certificate (ACME DNS-01). (podman) and an internal Caddy reverse proxy with a real wildcard certificate
(ACME DNS-01).
Addressing convention: a site owns `10.<siteId>.0.0/16`; VLAN `<id>` defaults Addressing convention: a site owns `10.<siteId>.0.0/16`; VLAN `<id>` defaults
to `10.<siteId>.<id>.0/24`, router at `.1`, DHCP pool `.100-.199`. The `mgmt` to `10.<siteId>.<id>.0/24`, router at `.1`, DHCP pool `.100-.199`. The `mgmt`
@@ -32,6 +33,7 @@ inventory.instances.router = {
wan.vlanId = 10; # or null for untagged PPPoE wan.vlanId = 10; # or null for untagged PPPoE
trunkPorts = [ "enp2s0" ]; trunkPorts = [ "enp2s0" ];
accessPorts.enp4s0 = "mgmt"; # untagged on-site recovery port accessPorts.enp4s0 = "mgmt"; # untagged on-site recovery port
# stagingPort = "enp3s0"; # DHCP uplink into the old LAN until cutover
vlans = { vlans = {
mgmt.id = 10; mgmt.id = 10;
lan.id = 20; lan.id = 20;
+5 -2
View File
@@ -1,12 +1,14 @@
# CrowdSec security engine + nftables bouncer: parses sshd auth attempts from # CrowdSec security engine + nftables bouncer: parses sshd auth attempts from
# the journal and bans offending source IPs at the firewall. Log-based (no # the journal and bans offending source IPs at the firewall. Log-based (no
# inline DPI) so it costs the N300 next to nothing. # inline DPI) so it costs the N300 next to nothing. Opt-in per site
# (`crowdsec.enable`): the hub sync needs internet at activation time.
{ settings }: { settings }:
{ ... }: { lib, ... }:
let let
cfg = settings; cfg = settings;
in in
{ {
config = lib.mkIf cfg.crowdsec.enable {
services.crowdsec = { services.crowdsec = {
enable = true; enable = true;
autoUpdateService = true; autoUpdateService = true;
@@ -43,4 +45,5 @@ in
registerBouncer.enable = true; registerBouncer.enable = true;
settings.mode = "nftables"; settings.mode = "nftables";
}; };
};
} }
+17 -1
View File
@@ -12,7 +12,19 @@ in
services.kea.dhcp4 = { services.kea.dhcp4 = {
enable = true; enable = true;
settings = { settings = {
interfaces-config.interfaces = lib.mapAttrsToList (name: _: "vlan-${name}") dhcpVlans; interfaces-config = {
interfaces = lib.mapAttrsToList (name: _: "vlan-${name}") dhcpVlans;
# The unit orders after network-online.target, which under networkd
# only waits for the WAN carrier (the vlan-* links are
# RequiredForOnline=no), so Kea can start before vlan-* have their
# addresses. By default it then logs the failed bind and runs with no
# socket at all: clients' DISCOVERs reach vlan-lan and nobody answers.
# Insist on every socket and keep retrying while networkd catches up;
# if it still cannot bind, exit and let systemd restart the unit.
service-sockets-require-all = true;
service-sockets-max-retries = 60;
service-sockets-retry-wait-time = 1000;
};
lease-database = { lease-database = {
type = "memfile"; type = "memfile";
persist = true; persist = true;
@@ -44,6 +56,10 @@ in
}; };
}; };
# The nixpkgs unit already has Restart=on-failure; space the restarts out so
# a persistent bind failure does not trip the start-rate limit.
systemd.services.kea-dhcp4-server.serviceConfig.RestartSec = 5;
services.blocky = { services.blocky = {
enable = true; enable = true;
settings = { settings = {
+36 -3
View File
@@ -3,6 +3,8 @@
# other VLANs -> DNS/DHCP on the router + WAN (if allowWan); no inter-VLAN # other VLANs -> DNS/DHCP on the router + WAN (if allowWan); no inter-VLAN
# WAN (ppp0) -> nothing inbound beyond established/related # WAN (ppp0) -> nothing inbound beyond established/related
# mesh -> admin SSH + metrics scrapes (same trust boundary as the fleet) # mesh -> admin SSH + metrics scrapes (same trust boundary as the fleet)
# staging -> admin SSH only inbound (pre-cutover uplink into the old LAN);
# allowWan VLANs are NATed out through it while ppp0 is down
{ settings }: { settings }:
{ lib, ... }: { lib, ... }:
let let
@@ -13,6 +15,13 @@ let
lib.filterAttrs (_: vlan: vlan.allowWan) cfg.vlans lib.filterAttrs (_: vlan: vlan.allowWan) cfg.vlans
); );
nonMgmtIfs = lib.filter (i: i != "vlan-mgmt") vlanIfs; nonMgmtIfs = lib.filter (i: i != "vlan-mgmt") vlanIfs;
# allowWan VLANs may also leave through the staging uplink. Same set as
# networking.nat.internalInterfaces below, so `allowWan` holds on both
# exits. The kernel picks the exit: ppp0 (metric 0, see pppoe.nix) while
# the session is up, the staging DHCP route (metric 1024) otherwise.
stagingExit = cfg.stagingPort != null && wanVlanIfs != [ ];
wanVlanSet = "{ ${lib.concatMapStringsSep ", " (i: ''"${i}"'') wanVlanIfs} }";
in in
{ {
networking.nftables.enable = true; networking.nftables.enable = true;
@@ -26,14 +35,20 @@ in
filterForward = true; filterForward = true;
trustedInterfaces = [ "vlan-mgmt" ]; trustedInterfaces = [ "vlan-mgmt" ];
# Non-mgmt VLANs may only talk to the router's DNS and DHCP. # Non-mgmt VLANs may only talk to the router's DNS and DHCP; the staging
interfaces = lib.genAttrs nonMgmtIfs (_: { # uplink (old LAN, pre-cutover) gets admin SSH so the box can be reached
# before the mgmt VLAN or the mesh are up.
interfaces =
lib.genAttrs nonMgmtIfs (_: {
allowedTCPPorts = [ 53 ]; allowedTCPPorts = [ 53 ];
allowedUDPPorts = [ allowedUDPPorts = [
53 53
67 67
]; ];
}); })
// lib.optionalAttrs (cfg.stagingPort != null) {
${cfg.stagingPort}.allowedTCPPorts = [ 22 ];
};
extraInputRules = '' extraInputRules = ''
ip6 saddr ${cfg.mesh.subnet} tcp dport 22 accept comment "admin ssh over the mesh" ip6 saddr ${cfg.mesh.subnet} tcp dport 22 accept comment "admin ssh over the mesh"
@@ -43,6 +58,9 @@ in
extraForwardRules = '' extraForwardRules = ''
tcp flags syn tcp option maxseg size set rt mtu comment "MSS clamp for PPPoE mtu 1492" tcp flags syn tcp option maxseg size set rt mtu comment "MSS clamp for PPPoE mtu 1492"
iifname "vlan-mgmt" accept comment "mgmt reaches all VLANs and the WAN" iifname "vlan-mgmt" accept comment "mgmt reaches all VLANs and the WAN"
''
+ lib.optionalString stagingExit ''
iifname ${wanVlanSet} oifname "${cfg.stagingPort}" accept comment "allowWan VLANs out via the staging uplink"
''; '';
}; };
@@ -55,4 +73,19 @@ in
externalInterface = "ppp0"; externalInterface = "ppp0";
internalInterfaces = wanVlanIfs; internalInterfaces = wanVlanIfs;
}; };
# networking.nat only masquerades on its single externalInterface; the
# staging uplink needs its own postrouting chain (nixos-nat's is
# oifname-scoped to ppp0, so the two never both apply).
networking.nftables.tables = lib.optionalAttrs stagingExit {
router-staging-nat = {
family = "ip";
content = ''
chain post {
type nat hook postrouting priority srcnat;
iifname ${wanVlanSet} oifname "${cfg.stagingPort}" masquerade comment "allowWan VLANs out via the staging uplink"
}
'';
};
};
} }
+20
View File
@@ -250,6 +250,24 @@ in
''; '';
}; };
stagingPort = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "enp3s0";
description = ''
Temporary DHCPv4-client uplink into the existing LAN while the box
runs alongside the router it replaces: gives it (and, NATed, the
allowWan VLANs) internet + mesh before the WAN port is cabled; once
the PPPoE session is up its default route wins, and the staging
route only carries traffic again if the session drops (PPPoE simply
retries until then). The port is in no VLAN zone; inbound, the
firewall admits only SSH on it. Do NOT connect the trunk ports to
the production switch while staging Kea on the mgmt tag would
fight the old router's DHCP in one broadcast domain. Set to null at
cutover (and usually hand the port back to `trunkPorts`).
'';
};
vlans = lib.mkOption { vlans = lib.mkOption {
type = lib.types.attrsOf (lib.types.submodule vlanModule); type = lib.types.attrsOf (lib.types.submodule vlanModule);
description = "VLANs served at this site; `mgmt` and `lan` are mandatory."; description = "VLANs served at this site; `mgmt` and `lan` are mandatory.";
@@ -267,6 +285,8 @@ in
omada.enable = lib.mkEnableOption "TP-Link Omada SDN controller (podman container)"; omada.enable = lib.mkEnableOption "TP-Link Omada SDN controller (podman container)";
crowdsec.enable = lib.mkEnableOption "CrowdSec (sshd log parsing) with the nftables bouncer";
proxy = { proxy = {
enable = lib.mkEnableOption "internal reverse proxy (Caddy, wildcard cert via DNS-01)"; enable = lib.mkEnableOption "internal reverse proxy (Caddy, wildcard cert via DNS-01)";
+19 -1
View File
@@ -21,6 +21,16 @@ in
assertion = lib.all (p: !(cfg.accessPorts ? ${p})) cfg.trunkPorts; assertion = lib.all (p: !(cfg.accessPorts ? ${p})) cfg.trunkPorts;
message = "router: a port cannot be both a trunk and an access port."; message = "router: a port cannot be both a trunk and an access port.";
} }
{
assertion =
cfg.stagingPort == null
|| !(
cfg.stagingPort == cfg.wan.interface
|| lib.elem cfg.stagingPort cfg.trunkPorts
|| cfg.accessPorts ? ${cfg.stagingPort}
);
message = "router: stagingPort ${toString cfg.stagingPort} is also the WAN, a trunk or an access port.";
}
]; ];
# Router diagnostics toolkit: packets (tcpdump), path (mtr), link # Router diagnostics toolkit: packets (tcpdump), path (mtr), link
@@ -73,7 +83,15 @@ in
let let
taggedAll = lib.mapAttrsToList (_: vlan: { VLAN = vlan.id; }) cfg.vlans; taggedAll = lib.mapAttrsToList (_: vlan: { VLAN = vlan.id; }) cfg.vlans;
in in
{ lib.optionalAttrs (cfg.stagingPort != null) {
# Staging uplink (see interface.nix): plain DHCPv4 client on a spare
# port, no bridge/VLAN membership, so the firewall treats it as untrusted.
"05-staging" = {
matchConfig.Name = cfg.stagingPort;
networkConfig.DHCP = "ipv4";
};
}
// {
# WAN port carries only the PPPoE session; no IP config of its own. # WAN port carries only the PPPoE session; no IP config of its own.
"10-wan" = { "10-wan" = {
matchConfig.Name = cfg.wan.interface; matchConfig.Name = cfg.wan.interface;
+6
View File
@@ -30,6 +30,11 @@ in
''; '';
}; };
# defaultroute-metric 0: pppd refuses `defaultroute` while any other
# default route exists (e.g. the staging uplink's DHCP route, metric 1024,
# network.nix) unless given a metric; with 0 it only checks for a metric-0
# route, installs its own as the preferred exit, and removes it again on
# hangup so the staging route takes over.
services.pppd = { services.pppd = {
enable = true; enable = true;
peers.wan = { peers.wan = {
@@ -40,6 +45,7 @@ in
file ${creds.files."user-opts".path} file ${creds.files."user-opts".path}
noipdefault noipdefault
defaultroute defaultroute
defaultroute-metric 0
noauth noauth
hide-password hide-password
persist persist
+60 -7
View File
@@ -1,14 +1,17 @@
# End-to-end VM test of the router service: a PPPoE access concentrator plays # End-to-end VM test of the router service: a PPPoE access concentrator plays
# the ISP on the WAN port, a trunk carries tagged lan/iot VLANs to `client`, # the ISP on the WAN port, a trunk carries tagged lan/iot VLANs to `client`,
# and an untagged access port carries mgmt to `admin`. # an untagged access port carries mgmt to `admin`, and `oldlan` is the DHCP
# network the box is staged in before cutover.
# #
# isp ---(vlan 1: PPPoE)--- wan [gw] trunk ---(vlan 2: tagged 20/40)--- client # isp ---(vlan 1: PPPoE)--- wan [gw] trunk ---(vlan 2: tagged 20/40)--- client
# access --(vlan 3: untagged mgmt)--- admin # access --(vlan 3: untagged mgmt)--- admin
# staging -(vlan 4: DHCP client)--- oldlan
# #
# What is proven: PPPoE dial-in with the vars-provided credentials, bridge # What is proven: PPPoE dial-in with the vars-provided credentials, bridge
# VLAN tagging/untagging, Kea leases and reservations per VLAN, Blocky # VLAN tagging/untagging, Kea leases and reservations per VLAN, Blocky
# answering on the VLAN with the blocklist active, NAT to the WAN, and the # answering on the VLAN with the blocklist active, NAT to the WAN, the
# firewall trust model (allowWan, mgmt-only SSH, no inter-VLAN forwarding). # firewall trust model (allowWan, mgmt-only SSH, no inter-VLAN forwarding),
# and the staging uplink as NATed fallback exit behind ppp0.
{ pkgs, lib, ... }: { pkgs, lib, ... }:
let let
# The vars mock answers every prompt with "mock-prompt-value-<name>"; the # The vars mock answers every prompt with "mock-prompt-value-<name>"; the
@@ -20,6 +23,10 @@ let
clientAddress = "10.9.20.50"; clientAddress = "10.9.20.50";
adminMac = "02:00:00:00:00:10"; adminMac = "02:00:00:00:00:10";
adminAddress = "10.9.10.50"; adminAddress = "10.9.10.50";
oldlanAddress = "192.168.88.1";
# Only reachable through oldlan's router role, i.e. via gw's staging
# default route (metric 1024); ppp0's metric-0 default must win while up.
beyondStaging = "203.0.113.1";
in in
{ {
name = "router"; name = "router";
@@ -36,6 +43,7 @@ in
isp = { }; isp = { };
client = { }; client = { };
admin = { }; admin = { };
oldlan = { };
}; };
instances.router = { instances.router = {
@@ -48,6 +56,7 @@ in
wan.interface = "wan"; wan.interface = "wan";
trunkPorts = [ "trunk" ]; trunkPorts = [ "trunk" ];
accessPorts.access = "mgmt"; accessPorts.access = "mgmt";
stagingPort = "staging";
vlans = { vlans = {
mgmt = { mgmt = {
id = 10; id = 10;
@@ -111,6 +120,10 @@ in
vlan = 3; vlan = 3;
assignIP = false; assignIP = false;
}; };
staging = {
vlan = 4;
assignIP = false;
};
}; };
# Something must listen on 22 for the mgmt-only SSH rule to be observable # Something must listen on 22 for the mgmt-only SSH rule to be observable
@@ -118,13 +131,11 @@ in
services.openssh.enable = true; services.openssh.enable = true;
# The sandbox has no internet: serve the blocklist from a local file # The sandbox has no internet: serve the blocklist from a local file
# instead of GitHub, and skip CrowdSec, whose hub sync needs the network # instead of GitHub. (CrowdSec, whose hub sync needs the network too,
# (it is not what this test exercises). # is opt-in and stays off.)
services.blocky.settings.blocking.denylists.ads = lib.mkForce [ services.blocky.settings.blocking.denylists.ads = lib.mkForce [
(toString (pkgs.writeText "ads.hosts" "0.0.0.0 ads.example.com\n")) (toString (pkgs.writeText "ads.hosts" "0.0.0.0 ads.example.com\n"))
]; ];
services.crowdsec.enable = lib.mkForce false;
services.crowdsec-firewall-bouncer.enable = lib.mkForce false;
# Two simulated radios: wlan0 is the AP (settings above), wlan1 plays a # Two simulated radios: wlan0 is the AP (settings above), wlan1 plays a
# wireless client. It lives in its own network namespace, like the # wireless client. It lives in its own network namespace, like the
@@ -251,6 +262,32 @@ in
}; };
environment.systemPackages = [ pkgs.netcat ]; environment.systemPackages = [ pkgs.netcat ];
}; };
# The LAN the box is staged in: a DHCP server handing gw its uplink
# lease, plus an address that is only reachable via that uplink's
# default route. No route back to 10.9.0.0/16: replies only reach the
# clients if gw masquerades them.
oldlan = {
virtualisation.interfaces.staging = {
vlan = 4;
assignIP = false;
};
networking.useDHCP = false;
networking.useNetworkd = true;
systemd.network.networks."10-staging" = {
matchConfig.Name = "staging";
address = [
"${oldlanAddress}/24"
"${beyondStaging}/32"
];
networkConfig.DHCPServer = true;
dhcpServerConfig = {
PoolOffset = 100;
PoolSize = 50;
};
};
networking.firewall.allowedUDPPorts = [ 67 ];
};
}; };
testScript = '' testScript = ''
@@ -302,5 +339,21 @@ in
gw.wait_until_succeeds("ip netns exec sta wpa_cli -i wlan1 status | grep -q wpa_state=COMPLETED") gw.wait_until_succeeds("ip netns exec sta wpa_cli -i wlan1 status | grep -q wpa_state=COMPLETED")
gw.succeed("timeout 60 sta-dhcp") gw.succeed("timeout 60 sta-dhcp")
gw.succeed("ip netns exec sta ip -4 addr show wlan1 | grep -q 'inet 10.9.20.1[0-9][0-9]/24'") gw.succeed("ip netns exec sta ip -4 addr show wlan1 | grep -q 'inet 10.9.20.1[0-9][0-9]/24'")
with subtest("Staging uplink: NATed exit for allowWan VLANs, behind ppp0 while it is up"):
gw.wait_until_succeeds("ip -4 route show default dev staging | grep -q 'via ${oldlanAddress}'")
# pppd installs its default route despite the DHCP one (defaultroute-metric 0).
gw.succeed("ip route get ${beyondStaging} | grep -q 'dev ppp0'")
# On-link old-LAN hosts are reached through the staging port regardless.
client.succeed("ping -c1 -W2 -I lan0 ${oldlanAddress}")
client.fail("ping -c1 -W2 -I iot0 ${oldlanAddress}")
# ppp0 down: the staging route carries the WAN traffic, allowWan still holds.
gw.systemctl("stop pppd-wan.service")
gw.wait_until_succeeds("ip route get ${beyondStaging} | grep -q 'dev staging'")
client.succeed("ping -c1 -W2 -I lan0 ${beyondStaging}")
client.fail("ping -c1 -W2 -I iot0 ${beyondStaging}")
# ppp0 back: preferred again.
gw.systemctl("start pppd-wan.service")
gw.wait_until_succeeds("ip route get ${beyondStaging} | grep -q 'dev ppp0'")
''; '';
} }
+19 -18
View File
@@ -12,7 +12,6 @@
{ {
config, config,
lib, lib,
pkgs,
... ...
}: }:
let let
@@ -121,26 +120,18 @@ in
); );
message = "router: wifi.radios.<radio>.macAddress is required for radios broadcasting more than one network."; message = "router: wifi.radios.<radio>.macAddress is required for radios broadcasting more than one network.";
} }
{
assertion =
!config.networking.wireless.enable
&& !config.networking.wireless.iwd.enable
&& !config.networking.networkmanager.enable;
message = "router: wifi.enable needs the radios for hostapd; disable networking.wireless (wpa_supplicant), iwd and NetworkManager.";
}
]; ];
# Regulatory database for the kernel, so countryCode actually applies. # Regulatory database for the kernel, so countryCode actually applies.
hardware.wirelessRegulatoryDatabase = true; hardware.wirelessRegulatoryDatabase = true;
# The kernel refuses to bridge a wireless interface in station mode, and
# networkd stops retrying before hostapd switches the radio to AP mode;
# so put it in AP mode the moment it appears (kernel name or the renamed
# one, whichever the user configured). hostapd finds it already there.
services.udev.extraRules = lib.concatMapStrings (
radio:
let
run = ''RUN+="${pkgs.iw}/bin/iw dev ${radio} set type __ap"'';
in
''
ACTION=="add", SUBSYSTEM=="net", KERNEL=="${radio}", ${run}
ACTION=="add", SUBSYSTEM=="net", NAME=="${radio}", ${run}
''
) (lib.attrNames wifi.radios);
clan.core.vars.generators = lib.genAttrs (map (name: "wifi-${name}-passphrase") secured) ( clan.core.vars.generators = lib.genAttrs (map (name: "wifi-${name}-passphrase") secured) (
gen: gen:
let let
@@ -184,12 +175,22 @@ in
}; };
# Each BSS is an untagged access port of its VLAN on br0 (cf. accessPorts # Each BSS is an untagged access port of its VLAN on br0 (cf. accessPorts
# in network.nix); networkd enslaves the interface once hostapd creates it. # in network.nix). The kernel refuses to bridge a wireless interface in
# station mode (IFF_DONT_BRIDGE), and a failed enslave parks the link in
# networkd's `failed` state for good: networkd only re-evaluates a link
# when its matching .network file changes. So match on the AP interface
# type as well as the name: the radio's initial station-mode netdev
# matches nothing (unmanaged), and once hostapd switches it to AP and
# brings the carrier up, networkd matches this file for the first time
# and enslaves it. Extra BSSes are created by hostapd in AP mode already.
systemd.network.networks = lib.listToAttrs ( systemd.network.networks = lib.listToAttrs (
map ( map (
b: b:
lib.nameValuePair "27-wifi-${b.iface}" { lib.nameValuePair "27-wifi-${b.iface}" {
matchConfig.Name = b.iface; matchConfig = {
Name = b.iface;
WLANInterfaceType = "ap";
};
networkConfig.Bridge = "br0"; networkConfig.Bridge = "br0";
bridgeVLANs = [ bridgeVLANs = [
{ {