Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 kurogeek
|
c0c2193429 | ||
|
kurogeek
|
63e8b6252c | ||
|
kurogeek
|
a3705e7a93 |
+17
-17
@@ -20,22 +20,22 @@ mgmt-only trust model, SSH exposure and the Wi-Fi bridge ports. Run it with
|
||||
|
||||
## What each gateway runs
|
||||
|
||||
| Function | Implementation |
|
||||
| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| WAN | PPPoE (`pppd`), per-site ISP credentials via clan vars prompts; `wan.vlanId` when the ISP tags the session (AIS: 10); `wan.macAddress` to clone the old router's MAC if the ISP has it pinned |
|
||||
| LAN | VLAN-filtering bridge `br0` over the trunk ports (networkd); `accessPorts` pin a port untagged to one VLAN — convention: the last copper port is an untagged `mgmt` recovery port; `stagingPort` turns a spare port into a DHCP uplink into the old LAN while the box runs alongside the router it replaces (drop at cutover) |
|
||||
| Firewall/NAT | nftables: default-deny WAN, no inter-VLAN, MSS clamp, v4 NAT |
|
||||
| DHCP | Kea, one subnet per VLAN |
|
||||
| DNS | Blocky (blocklist resolver), metrics on :4000 scraped by control |
|
||||
| IPv6 | DHCPv6-PD on ppp0, /64 per VLAN via SLAAC |
|
||||
| Bans | Optional per site (`crowdsec.enable`): CrowdSec + nftables bouncer (sshd log parsing) |
|
||||
| Omada | Optional per site: TP-Link Omada controller as a podman container |
|
||||
| Wi-Fi | Optional: hostapd on the router's radios; each SSID (`wifi.networks`) is an untagged access port of its VLAN, passphrases via vars prompts — see `modules/clan/router/README.md` |
|
||||
| Proxy | Optional: Caddy reverse proxy for internal services under `*.<site><n>.cnx.network` with a real Let's Encrypt wildcard (DNS-01 against ns1) |
|
||||
| Diagnostics | iperf3 server on 5201, reachable from every VLAN and the mesh (`iperf3 -c <router-vlan-addr>`); CLI toolkit: tcpdump, mtr, ethtool, conntrack, kdig, iftop, librespeed-cli |
|
||||
| Speed test | Hourly librespeed run (`speedtest.timer`) → `speedtest_*` metrics via node_exporter; vmalert flags download < 50% of the link's own 7-day median (`WANSpeedDegraded`) |
|
||||
| Management | ZeroTier mesh: SSH, node_exporter, journald upload — like the fleet |
|
||||
| Storage | Single-disk ZFS (zstd, `/var` its own dataset for snapshots) |
|
||||
| Function | Implementation |
|
||||
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| WAN | PPPoE (`pppd`), per-site ISP credentials via clan vars prompts; `wan.vlanId` when the ISP tags the session (AIS: 10); `wan.macAddress` to clone the old router's MAC if the ISP has it pinned |
|
||||
| LAN | VLAN-filtering bridge `br0` over the trunk ports (networkd); `accessPorts` pin a port untagged to one VLAN — convention: the last copper port is an untagged `mgmt` recovery port |
|
||||
| Firewall/NAT | nftables: default-deny WAN, no inter-VLAN, MSS clamp, v4 NAT |
|
||||
| DHCP | Kea, one subnet per VLAN |
|
||||
| DNS | Blocky (blocklist resolver), metrics on :4000 scraped by control |
|
||||
| IPv6 | DHCPv6-PD on ppp0, /64 per VLAN via SLAAC |
|
||||
| Bans | CrowdSec + nftables bouncer (sshd log parsing) |
|
||||
| Omada | Optional per site: TP-Link Omada controller as a podman container |
|
||||
| Wi-Fi | Optional: hostapd on the router's radios; each SSID (`wifi.networks`) is an untagged access port of its VLAN, passphrases via vars prompts — see `modules/clan/router/README.md` |
|
||||
| Proxy | Optional: Caddy reverse proxy for internal services under `*.<site><n>.cnx.network` with a real Let's Encrypt wildcard (DNS-01 against ns1) |
|
||||
| Diagnostics | iperf3 server on 5201, reachable from every VLAN and the mesh (`iperf3 -c <router-vlan-addr>`); CLI toolkit: tcpdump, mtr, ethtool, conntrack, kdig, iftop, librespeed-cli |
|
||||
| Speed test | Hourly librespeed run (`speedtest.timer`) → `speedtest_*` metrics via node_exporter; vmalert flags download < 50% of the link's own 7-day median (`WANSpeedDegraded`) |
|
||||
| Management | ZeroTier mesh: SSH, node_exporter, journald upload — like the fleet |
|
||||
| Storage | Single-disk ZFS (zstd, `/var` its own dataset for snapshots) |
|
||||
|
||||
## Addressing plan
|
||||
|
||||
@@ -71,7 +71,7 @@ Trust model: mgmt → everything; other VLANs → router DNS/DHCP + internet onl
|
||||
installer: `ls -l /dev/disk/by-id/`).
|
||||
2. Add the machine to `inventory.machines` in `clan.nix`, to the `router`
|
||||
instance in `inventory.nix` (`roles.default.machines.gw-<city>-<n>.settings`: `site`,
|
||||
`siteId` (next free number), port names, VLANs, `omada.enable`, `crowdsec.enable`; keep the
|
||||
`siteId` (next free number), port names, VLANs, `omada.enable`; keep the
|
||||
`mgmt`/`lan` VLANs), and to the machine list in `modules/mesh-hosts.nix`.
|
||||
Do **not** add it to `modules/hosts.nix` (dynamic PPPoE IP; clan connects
|
||||
over the mesh).
|
||||
|
||||
+4
-7
@@ -40,11 +40,10 @@ in
|
||||
wan.interface = "enp1s0";
|
||||
wan.vlanId = null; # this ISP runs PPPoE untagged on the port
|
||||
wan.macAddress = "a8:b8:e0:01:06:87";
|
||||
trunkPorts = [ "enp2s0" ];
|
||||
# STAGING (remove at cutover, move enp3s0 back into trunkPorts):
|
||||
# DHCP-client uplink into the existing OPNsense LAN so the box has
|
||||
# internet + mesh while it runs alongside the old router.
|
||||
stagingPort = "enp3s0";
|
||||
trunkPorts = [
|
||||
"enp2s0"
|
||||
# "enp3s0" # STAGING: serves as the uplink until cutover (see the machine config)
|
||||
];
|
||||
# Dedicated on-site recovery port: untagged mgmt, always available even
|
||||
# if the switch config is broken.
|
||||
accessPorts.enp4s0 = "mgmt";
|
||||
@@ -77,8 +76,6 @@ in
|
||||
};
|
||||
# This site runs the Omada controller for its APs/switches.
|
||||
omada.enable = true;
|
||||
# sshd ban engine (was unconditional before the option existed).
|
||||
crowdsec.enable = true;
|
||||
|
||||
# Internal reverse proxy: real wildcard cert *.cnx1.cnx.network; Blocky
|
||||
# resolves the names to the router's LAN address for mgmt+lan clients.
|
||||
|
||||
@@ -16,6 +16,18 @@
|
||||
builtins.hashString "sha256" config.networking.hostName
|
||||
);
|
||||
|
||||
# STAGING (remove at cutover, and restore enp3s0 to trunkPorts in clan.nix):
|
||||
# DHCP-client uplink into the existing OPNsense LAN so the box has internet +
|
||||
# mesh while it runs alongside the old router. Default-deny firewall on this
|
||||
# interface (it's in no VLAN zone); PPPoE simply retries until the WAN port
|
||||
# is cabled. Do NOT connect the trunk ports to the production switch while
|
||||
# staging — Kea on tag 10 would fight the OPNsense LAN DHCP in one broadcast
|
||||
# domain.
|
||||
systemd.network.networks."05-staging" = {
|
||||
matchConfig.Name = "enp3s0";
|
||||
networkConfig.DHCP = "ipv4";
|
||||
};
|
||||
|
||||
time.timeZone = "Etc/GMT-7"; # UTC+7 (Thailand, fixed offset, no DST)
|
||||
services.chrony.enable = true;
|
||||
}
|
||||
|
||||
@@ -3,11 +3,10 @@
|
||||
Turns a machine with several NICs into a site gateway: PPPoE WAN (ISP
|
||||
credentials via vars prompts), a VLAN-filtering bridge over the LAN ports with
|
||||
one L3 interface per VLAN, Kea DHCP and Blocky DNS per VLAN, nftables
|
||||
firewall/NAT, DHCPv6-PD, an iperf3 server and a WAN speed-test timer.
|
||||
Optional: a Wi-Fi access point on the router's own radios (hostapd), CrowdSec
|
||||
with the nftables bouncer (sshd log parsing), the TP-Link Omada controller
|
||||
(podman) and an internal Caddy reverse proxy with a real wildcard certificate
|
||||
(ACME DNS-01).
|
||||
firewall/NAT, DHCPv6-PD, CrowdSec, an iperf3 server and a WAN speed-test
|
||||
timer. Optional: a Wi-Fi access point on the router's own radios (hostapd),
|
||||
the TP-Link Omada controller (podman) and an internal Caddy reverse proxy
|
||||
with a real wildcard certificate (ACME DNS-01).
|
||||
|
||||
Addressing convention: a site owns `10.<siteId>.0.0/16`; VLAN `<id>` defaults
|
||||
to `10.<siteId>.<id>.0/24`, router at `.1`, DHCP pool `.100-.199`. The `mgmt`
|
||||
@@ -33,7 +32,6 @@ inventory.instances.router = {
|
||||
wan.vlanId = 10; # or null for untagged PPPoE
|
||||
trunkPorts = [ "enp2s0" ];
|
||||
accessPorts.enp4s0 = "mgmt"; # untagged on-site recovery port
|
||||
# stagingPort = "enp3s0"; # DHCP uplink into the old LAN until cutover
|
||||
vlans = {
|
||||
mgmt.id = 10;
|
||||
lan.id = 20;
|
||||
|
||||
@@ -1,49 +1,46 @@
|
||||
# CrowdSec security engine + nftables bouncer: parses sshd auth attempts from
|
||||
# the journal and bans offending source IPs at the firewall. Log-based (no
|
||||
# inline DPI) so it costs the N300 next to nothing. Opt-in per site
|
||||
# (`crowdsec.enable`): the hub sync needs internet at activation time.
|
||||
# inline DPI) so it costs the N300 next to nothing.
|
||||
{ settings }:
|
||||
{ lib, ... }:
|
||||
{ ... }:
|
||||
let
|
||||
cfg = settings;
|
||||
in
|
||||
{
|
||||
config = lib.mkIf cfg.crowdsec.enable {
|
||||
services.crowdsec = {
|
||||
enable = true;
|
||||
autoUpdateService = true;
|
||||
hub.collections = [
|
||||
"crowdsecurity/linux"
|
||||
"crowdsecurity/sshd"
|
||||
services.crowdsec = {
|
||||
enable = true;
|
||||
autoUpdateService = true;
|
||||
hub.collections = [
|
||||
"crowdsecurity/linux"
|
||||
"crowdsecurity/sshd"
|
||||
];
|
||||
localConfig = {
|
||||
acquisitions = [
|
||||
{
|
||||
source = "journalctl";
|
||||
journalctl_filter = [ "_SYSTEMD_UNIT=sshd.service" ];
|
||||
labels.type = "syslog";
|
||||
}
|
||||
];
|
||||
# Never ban the ZeroTier mesh — it is the only admin path to these
|
||||
# boxes (no public SSH), so a false positive would lock us out.
|
||||
# Parser-stage whitelist: mesh events are dropped before any scenario.
|
||||
parsers.s02Enrich = [
|
||||
{
|
||||
name = "cnx/mesh-whitelist";
|
||||
description = "Whitelist the ZeroTier management mesh";
|
||||
whitelist = {
|
||||
reason = "ZeroTier mesh is the admin path";
|
||||
cidr = [ cfg.mesh.subnet ];
|
||||
};
|
||||
}
|
||||
];
|
||||
localConfig = {
|
||||
acquisitions = [
|
||||
{
|
||||
source = "journalctl";
|
||||
journalctl_filter = [ "_SYSTEMD_UNIT=sshd.service" ];
|
||||
labels.type = "syslog";
|
||||
}
|
||||
];
|
||||
# Never ban the ZeroTier mesh — it is the only admin path to these
|
||||
# boxes (no public SSH), so a false positive would lock us out.
|
||||
# Parser-stage whitelist: mesh events are dropped before any scenario.
|
||||
parsers.s02Enrich = [
|
||||
{
|
||||
name = "cnx/mesh-whitelist";
|
||||
description = "Whitelist the ZeroTier management mesh";
|
||||
whitelist = {
|
||||
reason = "ZeroTier mesh is the admin path";
|
||||
cidr = [ cfg.mesh.subnet ];
|
||||
};
|
||||
}
|
||||
];
|
||||
};
|
||||
};
|
||||
|
||||
services.crowdsec-firewall-bouncer = {
|
||||
enable = true;
|
||||
registerBouncer.enable = true;
|
||||
settings.mode = "nftables";
|
||||
};
|
||||
};
|
||||
|
||||
services.crowdsec-firewall-bouncer = {
|
||||
enable = true;
|
||||
registerBouncer.enable = true;
|
||||
settings.mode = "nftables";
|
||||
};
|
||||
}
|
||||
|
||||
@@ -12,19 +12,7 @@ in
|
||||
services.kea.dhcp4 = {
|
||||
enable = true;
|
||||
settings = {
|
||||
interfaces-config = {
|
||||
interfaces = lib.mapAttrsToList (name: _: "vlan-${name}") dhcpVlans;
|
||||
# The unit orders after network-online.target, which under networkd
|
||||
# only waits for the WAN carrier (the vlan-* links are
|
||||
# RequiredForOnline=no), so Kea can start before vlan-* have their
|
||||
# addresses. By default it then logs the failed bind and runs with no
|
||||
# socket at all: clients' DISCOVERs reach vlan-lan and nobody answers.
|
||||
# Insist on every socket and keep retrying while networkd catches up;
|
||||
# if it still cannot bind, exit and let systemd restart the unit.
|
||||
service-sockets-require-all = true;
|
||||
service-sockets-max-retries = 60;
|
||||
service-sockets-retry-wait-time = 1000;
|
||||
};
|
||||
interfaces-config.interfaces = lib.mapAttrsToList (name: _: "vlan-${name}") dhcpVlans;
|
||||
lease-database = {
|
||||
type = "memfile";
|
||||
persist = true;
|
||||
@@ -56,10 +44,6 @@ in
|
||||
};
|
||||
};
|
||||
|
||||
# The nixpkgs unit already has Restart=on-failure; space the restarts out so
|
||||
# a persistent bind failure does not trip the start-rate limit.
|
||||
systemd.services.kea-dhcp4-server.serviceConfig.RestartSec = 5;
|
||||
|
||||
services.blocky = {
|
||||
enable = true;
|
||||
settings = {
|
||||
|
||||
@@ -3,8 +3,6 @@
|
||||
# other VLANs -> DNS/DHCP on the router + WAN (if allowWan); no inter-VLAN
|
||||
# WAN (ppp0) -> nothing inbound beyond established/related
|
||||
# mesh -> admin SSH + metrics scrapes (same trust boundary as the fleet)
|
||||
# staging -> admin SSH only inbound (pre-cutover uplink into the old LAN);
|
||||
# allowWan VLANs are NATed out through it while ppp0 is down
|
||||
{ settings }:
|
||||
{ lib, ... }:
|
||||
let
|
||||
@@ -15,13 +13,6 @@ let
|
||||
lib.filterAttrs (_: vlan: vlan.allowWan) cfg.vlans
|
||||
);
|
||||
nonMgmtIfs = lib.filter (i: i != "vlan-mgmt") vlanIfs;
|
||||
|
||||
# allowWan VLANs may also leave through the staging uplink. Same set as
|
||||
# networking.nat.internalInterfaces below, so `allowWan` holds on both
|
||||
# exits. The kernel picks the exit: ppp0 (metric 0, see pppoe.nix) while
|
||||
# the session is up, the staging DHCP route (metric 1024) otherwise.
|
||||
stagingExit = cfg.stagingPort != null && wanVlanIfs != [ ];
|
||||
wanVlanSet = "{ ${lib.concatMapStringsSep ", " (i: ''"${i}"'') wanVlanIfs} }";
|
||||
in
|
||||
{
|
||||
networking.nftables.enable = true;
|
||||
@@ -35,20 +26,14 @@ in
|
||||
filterForward = true;
|
||||
trustedInterfaces = [ "vlan-mgmt" ];
|
||||
|
||||
# Non-mgmt VLANs may only talk to the router's DNS and DHCP; the staging
|
||||
# uplink (old LAN, pre-cutover) gets admin SSH so the box can be reached
|
||||
# before the mgmt VLAN or the mesh are up.
|
||||
interfaces =
|
||||
lib.genAttrs nonMgmtIfs (_: {
|
||||
allowedTCPPorts = [ 53 ];
|
||||
allowedUDPPorts = [
|
||||
53
|
||||
67
|
||||
];
|
||||
})
|
||||
// lib.optionalAttrs (cfg.stagingPort != null) {
|
||||
${cfg.stagingPort}.allowedTCPPorts = [ 22 ];
|
||||
};
|
||||
# Non-mgmt VLANs may only talk to the router's DNS and DHCP.
|
||||
interfaces = lib.genAttrs nonMgmtIfs (_: {
|
||||
allowedTCPPorts = [ 53 ];
|
||||
allowedUDPPorts = [
|
||||
53
|
||||
67
|
||||
];
|
||||
});
|
||||
|
||||
extraInputRules = ''
|
||||
ip6 saddr ${cfg.mesh.subnet} tcp dport 22 accept comment "admin ssh over the mesh"
|
||||
@@ -58,9 +43,6 @@ in
|
||||
extraForwardRules = ''
|
||||
tcp flags syn tcp option maxseg size set rt mtu comment "MSS clamp for PPPoE mtu 1492"
|
||||
iifname "vlan-mgmt" accept comment "mgmt reaches all VLANs and the WAN"
|
||||
''
|
||||
+ lib.optionalString stagingExit ''
|
||||
iifname ${wanVlanSet} oifname "${cfg.stagingPort}" accept comment "allowWan VLANs out via the staging uplink"
|
||||
'';
|
||||
};
|
||||
|
||||
@@ -73,19 +55,4 @@ in
|
||||
externalInterface = "ppp0";
|
||||
internalInterfaces = wanVlanIfs;
|
||||
};
|
||||
|
||||
# networking.nat only masquerades on its single externalInterface; the
|
||||
# staging uplink needs its own postrouting chain (nixos-nat's is
|
||||
# oifname-scoped to ppp0, so the two never both apply).
|
||||
networking.nftables.tables = lib.optionalAttrs stagingExit {
|
||||
router-staging-nat = {
|
||||
family = "ip";
|
||||
content = ''
|
||||
chain post {
|
||||
type nat hook postrouting priority srcnat;
|
||||
iifname ${wanVlanSet} oifname "${cfg.stagingPort}" masquerade comment "allowWan VLANs out via the staging uplink"
|
||||
}
|
||||
'';
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
@@ -250,24 +250,6 @@ in
|
||||
'';
|
||||
};
|
||||
|
||||
stagingPort = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "enp3s0";
|
||||
description = ''
|
||||
Temporary DHCPv4-client uplink into the existing LAN while the box
|
||||
runs alongside the router it replaces: gives it (and, NATed, the
|
||||
allowWan VLANs) internet + mesh before the WAN port is cabled; once
|
||||
the PPPoE session is up its default route wins, and the staging
|
||||
route only carries traffic again if the session drops (PPPoE simply
|
||||
retries until then). The port is in no VLAN zone; inbound, the
|
||||
firewall admits only SSH on it. Do NOT connect the trunk ports to
|
||||
the production switch while staging — Kea on the mgmt tag would
|
||||
fight the old router's DHCP in one broadcast domain. Set to null at
|
||||
cutover (and usually hand the port back to `trunkPorts`).
|
||||
'';
|
||||
};
|
||||
|
||||
vlans = lib.mkOption {
|
||||
type = lib.types.attrsOf (lib.types.submodule vlanModule);
|
||||
description = "VLANs served at this site; `mgmt` and `lan` are mandatory.";
|
||||
@@ -285,8 +267,6 @@ in
|
||||
|
||||
omada.enable = lib.mkEnableOption "TP-Link Omada SDN controller (podman container)";
|
||||
|
||||
crowdsec.enable = lib.mkEnableOption "CrowdSec (sshd log parsing) with the nftables bouncer";
|
||||
|
||||
proxy = {
|
||||
enable = lib.mkEnableOption "internal reverse proxy (Caddy, wildcard cert via DNS-01)";
|
||||
|
||||
|
||||
@@ -21,16 +21,6 @@ in
|
||||
assertion = lib.all (p: !(cfg.accessPorts ? ${p})) cfg.trunkPorts;
|
||||
message = "router: a port cannot be both a trunk and an access port.";
|
||||
}
|
||||
{
|
||||
assertion =
|
||||
cfg.stagingPort == null
|
||||
|| !(
|
||||
cfg.stagingPort == cfg.wan.interface
|
||||
|| lib.elem cfg.stagingPort cfg.trunkPorts
|
||||
|| cfg.accessPorts ? ${cfg.stagingPort}
|
||||
);
|
||||
message = "router: stagingPort ${toString cfg.stagingPort} is also the WAN, a trunk or an access port.";
|
||||
}
|
||||
];
|
||||
|
||||
# Router diagnostics toolkit: packets (tcpdump), path (mtr), link
|
||||
@@ -83,15 +73,7 @@ in
|
||||
let
|
||||
taggedAll = lib.mapAttrsToList (_: vlan: { VLAN = vlan.id; }) cfg.vlans;
|
||||
in
|
||||
lib.optionalAttrs (cfg.stagingPort != null) {
|
||||
# Staging uplink (see interface.nix): plain DHCPv4 client on a spare
|
||||
# port, no bridge/VLAN membership, so the firewall treats it as untrusted.
|
||||
"05-staging" = {
|
||||
matchConfig.Name = cfg.stagingPort;
|
||||
networkConfig.DHCP = "ipv4";
|
||||
};
|
||||
}
|
||||
// {
|
||||
{
|
||||
# WAN port carries only the PPPoE session; no IP config of its own.
|
||||
"10-wan" = {
|
||||
matchConfig.Name = cfg.wan.interface;
|
||||
|
||||
@@ -30,11 +30,6 @@ in
|
||||
'';
|
||||
};
|
||||
|
||||
# defaultroute-metric 0: pppd refuses `defaultroute` while any other
|
||||
# default route exists (e.g. the staging uplink's DHCP route, metric 1024,
|
||||
# network.nix) unless given a metric; with 0 it only checks for a metric-0
|
||||
# route, installs its own as the preferred exit, and removes it again on
|
||||
# hangup so the staging route takes over.
|
||||
services.pppd = {
|
||||
enable = true;
|
||||
peers.wan = {
|
||||
@@ -45,7 +40,6 @@ in
|
||||
file ${creds.files."user-opts".path}
|
||||
noipdefault
|
||||
defaultroute
|
||||
defaultroute-metric 0
|
||||
noauth
|
||||
hide-password
|
||||
persist
|
||||
|
||||
@@ -1,17 +1,14 @@
|
||||
# End-to-end VM test of the router service: a PPPoE access concentrator plays
|
||||
# the ISP on the WAN port, a trunk carries tagged lan/iot VLANs to `client`,
|
||||
# an untagged access port carries mgmt to `admin`, and `oldlan` is the DHCP
|
||||
# network the box is staged in before cutover.
|
||||
# and an untagged access port carries mgmt to `admin`.
|
||||
#
|
||||
# isp ---(vlan 1: PPPoE)--- wan [gw] trunk ---(vlan 2: tagged 20/40)--- client
|
||||
# access --(vlan 3: untagged mgmt)--- admin
|
||||
# staging -(vlan 4: DHCP client)--- oldlan
|
||||
#
|
||||
# What is proven: PPPoE dial-in with the vars-provided credentials, bridge
|
||||
# VLAN tagging/untagging, Kea leases and reservations per VLAN, Blocky
|
||||
# answering on the VLAN with the blocklist active, NAT to the WAN, the
|
||||
# firewall trust model (allowWan, mgmt-only SSH, no inter-VLAN forwarding),
|
||||
# and the staging uplink as NATed fallback exit behind ppp0.
|
||||
# answering on the VLAN with the blocklist active, NAT to the WAN, and the
|
||||
# firewall trust model (allowWan, mgmt-only SSH, no inter-VLAN forwarding).
|
||||
{ pkgs, lib, ... }:
|
||||
let
|
||||
# The vars mock answers every prompt with "mock-prompt-value-<name>"; the
|
||||
@@ -23,10 +20,6 @@ let
|
||||
clientAddress = "10.9.20.50";
|
||||
adminMac = "02:00:00:00:00:10";
|
||||
adminAddress = "10.9.10.50";
|
||||
oldlanAddress = "192.168.88.1";
|
||||
# Only reachable through oldlan's router role, i.e. via gw's staging
|
||||
# default route (metric 1024); ppp0's metric-0 default must win while up.
|
||||
beyondStaging = "203.0.113.1";
|
||||
in
|
||||
{
|
||||
name = "router";
|
||||
@@ -43,7 +36,6 @@ in
|
||||
isp = { };
|
||||
client = { };
|
||||
admin = { };
|
||||
oldlan = { };
|
||||
};
|
||||
|
||||
instances.router = {
|
||||
@@ -56,7 +48,6 @@ in
|
||||
wan.interface = "wan";
|
||||
trunkPorts = [ "trunk" ];
|
||||
accessPorts.access = "mgmt";
|
||||
stagingPort = "staging";
|
||||
vlans = {
|
||||
mgmt = {
|
||||
id = 10;
|
||||
@@ -120,10 +111,6 @@ in
|
||||
vlan = 3;
|
||||
assignIP = false;
|
||||
};
|
||||
staging = {
|
||||
vlan = 4;
|
||||
assignIP = false;
|
||||
};
|
||||
};
|
||||
|
||||
# Something must listen on 22 for the mgmt-only SSH rule to be observable
|
||||
@@ -131,11 +118,13 @@ in
|
||||
services.openssh.enable = true;
|
||||
|
||||
# The sandbox has no internet: serve the blocklist from a local file
|
||||
# instead of GitHub. (CrowdSec, whose hub sync needs the network too,
|
||||
# is opt-in and stays off.)
|
||||
# instead of GitHub, and skip CrowdSec, whose hub sync needs the network
|
||||
# (it is not what this test exercises).
|
||||
services.blocky.settings.blocking.denylists.ads = lib.mkForce [
|
||||
(toString (pkgs.writeText "ads.hosts" "0.0.0.0 ads.example.com\n"))
|
||||
];
|
||||
services.crowdsec.enable = lib.mkForce false;
|
||||
services.crowdsec-firewall-bouncer.enable = lib.mkForce false;
|
||||
|
||||
# Two simulated radios: wlan0 is the AP (settings above), wlan1 plays a
|
||||
# wireless client. It lives in its own network namespace, like the
|
||||
@@ -262,32 +251,6 @@ in
|
||||
};
|
||||
environment.systemPackages = [ pkgs.netcat ];
|
||||
};
|
||||
|
||||
# The LAN the box is staged in: a DHCP server handing gw its uplink
|
||||
# lease, plus an address that is only reachable via that uplink's
|
||||
# default route. No route back to 10.9.0.0/16: replies only reach the
|
||||
# clients if gw masquerades them.
|
||||
oldlan = {
|
||||
virtualisation.interfaces.staging = {
|
||||
vlan = 4;
|
||||
assignIP = false;
|
||||
};
|
||||
networking.useDHCP = false;
|
||||
networking.useNetworkd = true;
|
||||
systemd.network.networks."10-staging" = {
|
||||
matchConfig.Name = "staging";
|
||||
address = [
|
||||
"${oldlanAddress}/24"
|
||||
"${beyondStaging}/32"
|
||||
];
|
||||
networkConfig.DHCPServer = true;
|
||||
dhcpServerConfig = {
|
||||
PoolOffset = 100;
|
||||
PoolSize = 50;
|
||||
};
|
||||
};
|
||||
networking.firewall.allowedUDPPorts = [ 67 ];
|
||||
};
|
||||
};
|
||||
|
||||
testScript = ''
|
||||
@@ -339,21 +302,5 @@ in
|
||||
gw.wait_until_succeeds("ip netns exec sta wpa_cli -i wlan1 status | grep -q wpa_state=COMPLETED")
|
||||
gw.succeed("timeout 60 sta-dhcp")
|
||||
gw.succeed("ip netns exec sta ip -4 addr show wlan1 | grep -q 'inet 10.9.20.1[0-9][0-9]/24'")
|
||||
|
||||
with subtest("Staging uplink: NATed exit for allowWan VLANs, behind ppp0 while it is up"):
|
||||
gw.wait_until_succeeds("ip -4 route show default dev staging | grep -q 'via ${oldlanAddress}'")
|
||||
# pppd installs its default route despite the DHCP one (defaultroute-metric 0).
|
||||
gw.succeed("ip route get ${beyondStaging} | grep -q 'dev ppp0'")
|
||||
# On-link old-LAN hosts are reached through the staging port regardless.
|
||||
client.succeed("ping -c1 -W2 -I lan0 ${oldlanAddress}")
|
||||
client.fail("ping -c1 -W2 -I iot0 ${oldlanAddress}")
|
||||
# ppp0 down: the staging route carries the WAN traffic, allowWan still holds.
|
||||
gw.systemctl("stop pppd-wan.service")
|
||||
gw.wait_until_succeeds("ip route get ${beyondStaging} | grep -q 'dev staging'")
|
||||
client.succeed("ping -c1 -W2 -I lan0 ${beyondStaging}")
|
||||
client.fail("ping -c1 -W2 -I iot0 ${beyondStaging}")
|
||||
# ppp0 back: preferred again.
|
||||
gw.systemctl("start pppd-wan.service")
|
||||
gw.wait_until_succeeds("ip route get ${beyondStaging} | grep -q 'dev ppp0'")
|
||||
'';
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
{
|
||||
config,
|
||||
lib,
|
||||
pkgs,
|
||||
...
|
||||
}:
|
||||
let
|
||||
@@ -120,18 +121,26 @@ in
|
||||
);
|
||||
message = "router: wifi.radios.<radio>.macAddress is required for radios broadcasting more than one network.";
|
||||
}
|
||||
{
|
||||
assertion =
|
||||
!config.networking.wireless.enable
|
||||
&& !config.networking.wireless.iwd.enable
|
||||
&& !config.networking.networkmanager.enable;
|
||||
message = "router: wifi.enable needs the radios for hostapd; disable networking.wireless (wpa_supplicant), iwd and NetworkManager.";
|
||||
}
|
||||
];
|
||||
|
||||
# Regulatory database for the kernel, so countryCode actually applies.
|
||||
hardware.wirelessRegulatoryDatabase = true;
|
||||
|
||||
# The kernel refuses to bridge a wireless interface in station mode, and
|
||||
# networkd stops retrying before hostapd switches the radio to AP mode;
|
||||
# so put it in AP mode the moment it appears (kernel name or the renamed
|
||||
# one, whichever the user configured). hostapd finds it already there.
|
||||
services.udev.extraRules = lib.concatMapStrings (
|
||||
radio:
|
||||
let
|
||||
run = ''RUN+="${pkgs.iw}/bin/iw dev ${radio} set type __ap"'';
|
||||
in
|
||||
''
|
||||
ACTION=="add", SUBSYSTEM=="net", KERNEL=="${radio}", ${run}
|
||||
ACTION=="add", SUBSYSTEM=="net", NAME=="${radio}", ${run}
|
||||
''
|
||||
) (lib.attrNames wifi.radios);
|
||||
|
||||
clan.core.vars.generators = lib.genAttrs (map (name: "wifi-${name}-passphrase") secured) (
|
||||
gen:
|
||||
let
|
||||
@@ -175,22 +184,12 @@ in
|
||||
};
|
||||
|
||||
# Each BSS is an untagged access port of its VLAN on br0 (cf. accessPorts
|
||||
# in network.nix). The kernel refuses to bridge a wireless interface in
|
||||
# station mode (IFF_DONT_BRIDGE), and a failed enslave parks the link in
|
||||
# networkd's `failed` state for good: networkd only re-evaluates a link
|
||||
# when its matching .network file changes. So match on the AP interface
|
||||
# type as well as the name: the radio's initial station-mode netdev
|
||||
# matches nothing (unmanaged), and once hostapd switches it to AP and
|
||||
# brings the carrier up, networkd matches this file for the first time
|
||||
# and enslaves it. Extra BSSes are created by hostapd in AP mode already.
|
||||
# in network.nix); networkd enslaves the interface once hostapd creates it.
|
||||
systemd.network.networks = lib.listToAttrs (
|
||||
map (
|
||||
b:
|
||||
lib.nameValuePair "27-wifi-${b.iface}" {
|
||||
matchConfig = {
|
||||
Name = b.iface;
|
||||
WLANInterfaceType = "ap";
|
||||
};
|
||||
matchConfig.Name = b.iface;
|
||||
networkConfig.Bridge = "br0";
|
||||
bridgeVLANs = [
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user