12 Commits
Author SHA1 Message Date
kurogeek e316ec56ce flake: use flake-parts 2026-08-05 10:19:32 +07:00
kurogeek 559115e52b inventory.json: update install time of gw-cnx-1 2026-07-31 17:37:47 +07:00
kurogeek 9cfcf91b51 inventory.json: update install time of gw-cnx-1 2026-07-31 16:08:14 +07:00
kurogeek 570fe64497 inventory.json: update install time of gw-cnx-1 2026-07-31 15:09:07 +07:00
kurogeek 2ac4215237 inventory.json: update install time of gw-cnx-1 2026-07-31 13:03:43 +07:00
kurogeek d51a334ec6 inventory.json: update install time of gw-cnx-1 2026-07-31 13:00:57 +07:00
kurogeek b81c0aaa89 machines/gw-cnx-1/facter.json: update hardware configuration 2026-07-31 13:00:57 +07:00
Berwn fcd8e55024 Add untagged access ports and a staging uplink for gw-cnx-1
New cnx.router.accessPorts option pins a port untagged to one VLAN via
bridge PVID/EgressUntagged; convention is the last copper port as an
always-available mgmt recovery port. gw-cnx-1 port roles: enp1s0 WAN,
enp2s0 trunk, enp3s0 temporary DHCP uplink into the old OPNsense LAN
(back to trunk at cutover), enp4s0 untagged mgmt.
2026-07-31 12:57:34 +07:00
Berwn 19e1acda51 Renumber gw-cnx-1 to the fleet addressing scheme (OPNsense replacement)
The newedge.house site adopts the 10.1.<vlanId>.0/24 convention: mgmt 10
(the old untagged LAN), lan 20, iot 40, voip 50, dmz 60, unit1-5 110-150.
PPPoE is untagged at this site. Static leases move to a new per-VLAN
dhcp.reservations option rendered into Kea host reservations.
2026-07-31 11:39:54 +07:00
Berwn f5b6b4b55e Serve newedge.house from the fleet nameservers 2026-07-31 11:39:54 +07:00
Berwn 3646d43ac5 Add CNX WAN Speed dashboard for the gateway speed tests
Per-gateway download/upload/ping from the hourly librespeed runs, with
each link's 7-day median overlaid so the WANSpeedDegraded alert
condition is visible at a glance. Label-driven — new sites appear
automatically once scraped.
2026-07-31 11:03:31 +07:00
Berwn d8d4a686eb Add gateway diagnostics: iperf3, CLI toolkit, periodic WAN speed test
iperf3 serves throughput tests from every VLAN and the mesh; an hourly
librespeed run feeds speedtest_* metrics through node_exporter's textfile
collector, and vmalert flags download rates below half the link's own
7-day median so ISP degradation surfaces without per-site thresholds.
2026-07-31 10:44:52 +07:00
15 changed files with 4508 additions and 32 deletions
+14 -4
View File
@@ -13,7 +13,7 @@ Naming: `gw-<city>-<n>`, e.g. `gw-cnx-1`.
| Function | Implementation |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| WAN | PPPoE (`pppd`), per-site ISP credentials via clan vars prompts; `wan.vlanId` when the ISP tags the session (AIS: 10); `wan.macAddress` to clone the old router's MAC if the ISP has it pinned |
| LAN | VLAN-filtering bridge `br0` over the trunk ports (networkd) |
| LAN | VLAN-filtering bridge `br0` over the trunk ports (networkd); `accessPorts` pin a port untagged to one VLAN — convention: the last copper port is an untagged `mgmt` recovery port |
| Firewall/NAT | nftables: default-deny WAN, no inter-VLAN, MSS clamp, v4 NAT |
| DHCP | Kea, one subnet per VLAN |
| DNS | Blocky (blocklist resolver), metrics on :4000 scraped by control |
@@ -21,6 +21,8 @@ Naming: `gw-<city>-<n>`, e.g. `gw-cnx-1`.
| Bans | CrowdSec + nftables bouncer (sshd log parsing) |
| Omada | Optional per site: TP-Link Omada controller as a podman container |
| Proxy | Optional: Caddy reverse proxy for internal services under `*.<site><n>.cnx.network` with a real Let's Encrypt wildcard (DNS-01 against ns1) |
| Diagnostics | iperf3 server on 5201, reachable from every VLAN and the mesh (`iperf3 -c <router-vlan-addr>`); CLI toolkit: tcpdump, mtr, ethtool, conntrack, kdig, iftop, librespeed-cli |
| Speed test | Hourly librespeed run (`speedtest.timer`) → `speedtest_*` metrics via node_exporter; vmalert flags download < 50% of the link's own 7-day median (`WANSpeedDegraded`) |
| Management | ZeroTier mesh: SSH, node_exporter, journald upload — like the fleet |
| Storage | Single-disk ZFS (zstd, `/var` its own dataset for snapshots) |
@@ -40,9 +42,13 @@ should also shorten `dhcp.leaseTime` (default 86400 s) so the pool recycles.
First user: `gw-cnx-2` (site 2) runs the public WiFi — guest VLAN 30 at
`10.2.128.0/22`, pool `10.2.128.100 10.2.131.250`, `dhcp.leaseTime = 3600`.
| Site | siteId | mgmt | lan |
| ---- | ------ | -------------- | -------------- |
| cnx | 1 | `10.1.10.0/24` | `10.1.20.0/24` |
| Site | siteId | mgmt | lan | site-specific VLANs |
| ---- | ------ | -------------- | -------------- | --------------------------------------------------------------- |
| cnx | 1 | `10.1.10.0/24` | `10.1.20.0/24` | iot 40, voip 50, dmz 60, unit15 110/120/130/140/150 (all /24s) |
Static DHCP leases are declared per VLAN via `dhcp.reservations` (attribute
name = hostname, plus `hwAddress`/`ipAddress`); park them outside the pool,
in the `.2.99` infra range or `.200.254`.
Trust model: mgmt → everything; other VLANs → router DNS/DHCP + internet only
(no inter-VLAN); WAN → nothing inbound; mesh → SSH, metrics, Omada UI.
@@ -124,6 +130,10 @@ cnx.router.proxy = {
port's `bridgeVLANs` tagging and that the switch trunk carries the VLAN.
- **DNS/blocklist issues**: Blocky metrics are in VictoriaMetrics (job
`blocky`); `journalctl -u blocky` on the gateway.
- **Slow internet reported / `WANSpeedDegraded` firing**: check the
`speedtest_download_mbps` / `speedtest_ping_ms` trend in VictoriaMetrics,
then `librespeed-cli` and `mtr <isp-gw>` on the gateway to separate ISP
degradation from local saturation (`iftop` on ppp0 shows who's eating it).
- **Banned yourself**: `cscli decisions list` / `cscli decisions delete --ip <ip>`
on the gateway. The ZeroTier mesh `/88` is whitelisted at the parser stage
(`cnx/mesh-whitelist`), so admin access over the mesh can never be banned.
+4
View File
@@ -52,6 +52,10 @@ there is picked up):
- **CNX Uptime** (`uptime.json`) — per-host up/down status, current uptime,
availability over the selected window, and up/down history. Label-driven, so
every scraped host appears automatically.
- **CNX WAN Speed** (`wan.json`) — per-gateway WAN speed from the hourly
librespeed runs: latest download/upload/ping, throughput history with each
link's 7-day median as reference, latency/jitter, and test status. See
[Site gateways](./gateways.md).
- **parsedmarc** — DMARC aggregate/forensic report viewer. Auto-provisioned by
the `parsedmarc` module (not from `dashboards/`); reads its own Elasticsearch
datasource, not VictoriaMetrics. See [DMARC reports](#dmarc-reports) below.
Generated
+4
View File
@@ -247,6 +247,10 @@
"root": {
"inputs": {
"clan-core": "clan-core",
"flake-parts": [
"clan-core",
"flake-parts"
],
"nixos-mailserver": "nixos-mailserver",
"nixpkgs": [
"clan-core",
+25 -24
View File
@@ -7,13 +7,16 @@
inputs.nixos-mailserver.url = "gitlab:simple-nixos-mailserver/nixos-mailserver/nixos-26.05";
inputs.nixos-mailserver.inputs.nixpkgs.follows = "nixpkgs";
inputs.flake-parts.follows = "clan-core/flake-parts";
outputs =
{
inputs@{
self,
clan-core,
nixpkgs,
flake-parts,
...
}@inputs:
}:
let
# Usage see: https://docs.clan.lol
clan = clan-core.lib.clan {
@@ -36,38 +39,31 @@
"age-plugin-fido2-hmac"
];
};
in
flake-parts.lib.mkFlake { inherit inputs; } {
systems = [
"x86_64-linux"
"aarch64-linux"
"aarch64-darwin"
"x86_64-darwin"
];
forAllSystems = nixpkgs.lib.genAttrs systems;
pkgsFor = system: clan-core.inputs.nixpkgs.legacyPackages.${system};
treefmtFor = system: inputs.treefmt-nix.lib.evalModule (pkgsFor system) ./fmt.nix;
in
{
inherit (clan.config) nixosConfigurations nixosModules clanInternals;
clan = clan.config;
# `nix fmt` and the `nix flake check` formatting gate.
formatter = forAllSystems (system: (treefmtFor system).config.build.wrapper);
checks = forAllSystems (system: {
formatting = (treefmtFor system).config.build.check self;
});
flake = {
inherit (clan.config) nixosConfigurations nixosModules clanInternals;
clan = clan.config;
};
# Add the Clan cli tool to the dev shell.
# Use "nix develop" to enter the dev shell.
devShells = forAllSystems (
system:
perSystem =
{ system, ... }:
let
pkgs = clan-core.inputs.nixpkgs.legacyPackages.${system};
treefmtEval = inputs.treefmt-nix.lib.evalModule pkgs ./fmt.nix;
clanCli = clan-core.packages.${system}.clan-cli;
# `clan machines update a b c` normally runs machines in parallel,
# which interleaves their output and buries the YubiKey PIN prompts.
# This wrapper (first in PATH) runs them one at a time instead; any
# flags fall through to the real CLI untouched.
clanSequential = (pkgsFor system).writeShellScriptBin "clan" ''
clanSequential = pkgs.writeShellScriptBin "clan" ''
if [ "$#" -gt 3 ] && [ "$1" = machines ] && [ "$2" = update ]; then
shift 2
for arg in "$@"; do
@@ -84,14 +80,19 @@
'';
in
{
default = (pkgsFor system).mkShell {
# `nix fmt` and the `nix flake check` formatting gate.
formatter = treefmtEval.config.build.wrapper;
checks.formatting = treefmtEval.config.build.check self;
# Add the Clan cli tool to the dev shell.
# Use "nix develop" to enter the dev shell.
devShells.default = pkgs.mkShell {
packages = [
clanSequential
clanCli
(treefmtFor system).config.build.wrapper
treefmtEval.config.build.wrapper
];
};
}
);
};
};
}
+3
View File
@@ -14,6 +14,9 @@
},
"web01": {
"installedAt": 1781983723
},
"gw-cnx-1": {
"installedAt": 1785494267
}
}
}
+40 -4
View File
@@ -25,15 +25,40 @@
site = "cnx";
siteId = 1;
wan.interface = "enp1s0";
wan.vlanId = 10; # AIS delivers PPPoE tagged on VLAN 10
wan.vlanId = null; # this ISP runs PPPoE untagged on the port
trunkPorts = [
"enp2s0"
"enp3s0"
"enp4s0"
# "enp3s0" # STAGING: serves as the uplink below until cutover
];
# Dedicated on-site recovery port: untagged mgmt, always available even
# if the switch config is broken.
accessPorts.enp4s0 = "mgmt";
# Replaces the newedge.house OPNsense box; renumbered to the fleet
# convention (10.1.<vlanId>.0/24, router .1, pool .100-.199). The old
# untagged LAN becomes tagged mgmt — infra switch ports get PVID 10.
vlans = {
mgmt.id = 10; # 10.1.10.0/24 — APs, switches, Omada, admin
mgmt = {
id = 10; # 10.1.10.0/24 — servers, APs, switches, Omada, admin
dhcp.reservations.storinator01 = {
hwAddress = "7c:c2:55:e0:d6:40";
ipAddress = "10.1.10.53";
};
};
lan.id = 20; # 10.1.20.0/24 — trusted clients
iot.id = 40; # 10.1.40.0/24
voip.id = 50; # 10.1.50.0/24
dmz.id = 60; # 10.1.60.0/24
unit1.id = 110; # 10.1.110.0/24
unit2.id = 120; # 10.1.120.0/24
unit3.id = 130; # 10.1.130.0/24
unit4.id = 140; # 10.1.140.0/24
unit5 = {
id = 150; # 10.1.150.0/24
dhcp.reservations.newt = {
hwAddress = "7c:d3:0a:21:58:0b";
ipAddress = "10.1.150.22";
};
};
};
# This site runs the Omada controller for its APs/switches.
omada.enable = true;
@@ -50,6 +75,17 @@
};
};
# STAGING (remove at cutover, and restore enp3s0 to trunkPorts): DHCP-client
# uplink into the existing OPNsense LAN so the box has internet + mesh while
# it runs alongside the old router. Default-deny firewall on this interface
# (it's in no VLAN zone); PPPoE simply retries until the WAN port is cabled.
# Do NOT connect the trunk ports to the production switch while staging —
# Kea on tag 10 would fight the OPNsense LAN DHCP in one broadcast domain.
systemd.network.networks."05-staging" = {
matchConfig.Name = "enp3s0";
networkConfig.DHCP = "ipv4";
};
time.timeZone = "Etc/GMT-7"; # UTC+7 (Thailand, fixed offset, no DST)
services.chrony.enable = true;
}
File diff suppressed because it is too large Load Diff
+1
View File
@@ -4,4 +4,5 @@
"cnx.network"
"buildfor.life"
"cnx.email"
"newedge.house"
]
+13
View File
@@ -0,0 +1,13 @@
$ORIGIN newedge.house.
$TTL 3600
@ IN SOA ns1.cnx.network. hostmaster.cnx.network. (
2026061401 ; serial (ignored: Knot auto-assigns a dateserial on signing)
3600 ; refresh
900 ; retry
604800 ; expire
300 ) ; negative-cache TTL
; Served by the same nameservers (out-of-bailiwick, no glue needed here).
@ IN NS ns1.cnx.network.
@ IN NS ns2.cnx.network.
+25
View File
@@ -59,6 +59,31 @@ in
}
];
}
{
# WAN quality on the site gateways, fed by the hourly speedtest timer
# (modules/router/speedtest.nix) via the node_exporter textfile
# collector. Degradation is judged against the link's own 7-day median,
# so no per-site/per-ISP-plan threshold needs maintaining.
name = "wan";
rules = [
{
alert = "WANSpeedDegraded";
expr = "speedtest_download_mbps < 0.5 * quantile_over_time(0.5, speedtest_download_mbps[7d])";
for = "3h";
labels.severity = "warning";
annotations.summary = "WAN download on {{ $labels.instance }} is below half its 7-day median";
annotations.description = "Sustained for 3h (several test runs) likely ISP degradation, line fault, or saturation. Compare speedtest_upload_mbps/speedtest_ping_ms and run `librespeed-cli` manually on the gateway.";
}
{
alert = "WANSpeedTestFailing";
expr = "speedtest_success == 0";
for = "3h";
labels.severity = "warning";
annotations.summary = "Speed tests on {{ $labels.instance }} have been failing for 3h";
annotations.description = "librespeed-cli cannot complete a test while other traffic may still flow; check `journalctl -u speedtest` on the gateway.";
}
];
}
{
name = "backup";
rules = [
+267
View File
@@ -0,0 +1,267 @@
{
"uid": "cnx-wan",
"title": "CNX WAN Speed",
"tags": ["wan", "speedtest", "gateway", "cnx"],
"timezone": "browser",
"schemaVersion": 39,
"version": 1,
"refresh": "5m",
"time": { "from": "now-7d", "to": "now" },
"templating": { "list": [] },
"annotations": { "list": [] },
"panels": [
{
"type": "row",
"title": "WAN speed (site gateways)",
"id": 1,
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 0 }
},
{
"type": "stat",
"title": "Download (latest)",
"description": "Most recent hourly librespeed result per gateway.",
"id": 2,
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"gridPos": { "h": 5, "w": 6, "x": 0, "y": 1 },
"fieldConfig": {
"defaults": {
"unit": "bps",
"color": { "mode": "fixed", "fixedColor": "green" },
"noValue": "no data"
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": ["lastNotNull"],
"fields": "",
"values": false
},
"colorMode": "value",
"graphMode": "area",
"textMode": "value_and_name",
"orientation": "auto"
},
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"expr": "speedtest_download_mbps * 1e6",
"legendFormat": "{{instance}}",
"instant": true
}
]
},
{
"type": "stat",
"title": "Upload (latest)",
"description": "Most recent hourly librespeed result per gateway.",
"id": 3,
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"gridPos": { "h": 5, "w": 6, "x": 6, "y": 1 },
"fieldConfig": {
"defaults": {
"unit": "bps",
"color": { "mode": "fixed", "fixedColor": "blue" },
"noValue": "no data"
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": ["lastNotNull"],
"fields": "",
"values": false
},
"colorMode": "value",
"graphMode": "area",
"textMode": "value_and_name",
"orientation": "auto"
},
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"expr": "speedtest_upload_mbps * 1e6",
"legendFormat": "{{instance}}",
"instant": true
}
]
},
{
"type": "stat",
"title": "Ping (latest)",
"description": "Latency to the librespeed test server. Yellow over 50 ms, red over 100 ms.",
"id": 4,
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"gridPos": { "h": 5, "w": 6, "x": 12, "y": 1 },
"fieldConfig": {
"defaults": {
"unit": "ms",
"color": { "mode": "thresholds" },
"thresholds": {
"mode": "absolute",
"steps": [
{ "color": "green", "value": null },
{ "color": "yellow", "value": 50 },
{ "color": "red", "value": 100 }
]
},
"noValue": "no data"
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": ["lastNotNull"],
"fields": "",
"values": false
},
"colorMode": "value",
"graphMode": "none",
"textMode": "value_and_name",
"orientation": "auto"
},
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"expr": "speedtest_ping_ms",
"legendFormat": "{{instance}}",
"instant": true
}
]
},
{
"type": "stat",
"title": "Test status",
"description": "Whether the last speed test on each gateway completed. FAILING also fires the WANSpeedTestFailing alert after 3h.",
"id": 5,
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"gridPos": { "h": 5, "w": 6, "x": 18, "y": 1 },
"fieldConfig": {
"defaults": {
"color": { "mode": "thresholds" },
"thresholds": {
"mode": "absolute",
"steps": [{ "color": "green", "value": null }]
},
"noValue": "no data",
"mappings": [
{
"type": "value",
"options": {
"0": { "text": "FAILING", "color": "red", "index": 0 },
"1": { "text": "OK", "color": "green", "index": 1 }
}
}
]
},
"overrides": []
},
"options": {
"reduceOptions": {
"calcs": ["lastNotNull"],
"fields": "",
"values": false
},
"colorMode": "background",
"graphMode": "none",
"textMode": "value_and_name",
"orientation": "auto"
},
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"expr": "speedtest_success",
"legendFormat": "{{instance}}",
"instant": true
}
]
},
{
"type": "timeseries",
"title": "Download",
"description": "Hourly download rate per gateway, with each link's own 7-day median as reference — the WANSpeedDegraded alert fires when a link stays below half its median for 3h.",
"id": 6,
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"gridPos": { "h": 9, "w": 12, "x": 0, "y": 6 },
"fieldConfig": {
"defaults": { "unit": "bps", "min": 0 },
"overrides": [
{
"matcher": { "id": "byRegexp", "options": ".*7d median.*" },
"properties": [
{
"id": "custom.lineStyle",
"value": { "fill": "dash", "dash": [10, 10] }
},
{ "id": "custom.fillOpacity", "value": 0 }
]
}
]
},
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"expr": "speedtest_download_mbps * 1e6",
"legendFormat": "{{instance}}"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"expr": "quantile_over_time(0.5, speedtest_download_mbps[7d]) * 1e6",
"legendFormat": "{{instance}} 7d median"
}
]
},
{
"type": "timeseries",
"title": "Upload",
"description": "Hourly upload rate per gateway.",
"id": 7,
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"gridPos": { "h": 9, "w": 12, "x": 12, "y": 6 },
"fieldConfig": {
"defaults": { "unit": "bps", "min": 0 },
"overrides": []
},
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"expr": "speedtest_upload_mbps * 1e6",
"legendFormat": "{{instance}}"
}
]
},
{
"type": "timeseries",
"title": "Latency & jitter",
"description": "Ping and jitter towards the librespeed test server. A latency climb with stable throughput usually means bufferbloat or an ISP routing change.",
"id": 8,
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 15 },
"fieldConfig": {
"defaults": { "unit": "ms", "min": 0 },
"overrides": []
},
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"expr": "speedtest_ping_ms",
"legendFormat": "{{instance}} ping"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "victoriametrics" },
"expr": "speedtest_jitter_ms",
"legendFormat": "{{instance}} jitter"
}
]
}
]
}
+71
View File
@@ -11,6 +11,7 @@
{
config,
lib,
pkgs,
...
}:
let
@@ -67,6 +68,25 @@ let
e.g. public-WiFi guest VLANs (3600-7200), so the pool recycles.
'';
};
reservations = lib.mkOption {
type = lib.types.attrsOf (
lib.types.submodule {
options = {
hwAddress = lib.mkOption {
type = lib.types.str;
example = "aa:bb:cc:dd:ee:ff";
description = "Client MAC address.";
};
ipAddress = lib.mkOption {
type = lib.types.str;
description = "Fixed address handed to this client (inside the VLAN's subnet, outside the pool).";
};
};
}
);
default = { };
description = "Static DHCP leases; the attribute name becomes the client's hostname.";
};
};
allowWan = lib.mkOption {
type = lib.types.bool;
@@ -85,6 +105,8 @@ in
./crowdsec.nix
./omada.nix
./proxy.nix
./iperf.nix
./speedtest.nix
];
options.cnx.router = {
@@ -138,6 +160,19 @@ in
description = "LAN ports carrying all VLANs tagged (incl. any 10G SFP+ ports).";
};
accessPorts = lib.mkOption {
type = lib.types.attrsOf lib.types.str;
default = { };
example = {
enp4s0 = "mgmt";
};
description = ''
Ports acting as untagged access ports on a single VLAN (port name ->
VLAN name). Frames are untagged on the wire; the bridge tags them with
the VLAN's PVID. Use for an always-available on-site mgmt port.
'';
};
vlans = lib.mkOption {
type = lib.types.attrsOf (lib.types.submodule vlanModule);
description = "VLANs served at this site; `mgmt` and `lan` are mandatory.";
@@ -150,6 +185,27 @@ in
assertion = cfg.vlans ? mgmt && cfg.vlans ? lan;
message = "cnx.router: every site must define the `mgmt` and `lan` VLANs.";
}
{
assertion = lib.all (v: cfg.vlans ? ${v}) (lib.attrValues cfg.accessPorts);
message = "cnx.router: every accessPorts value must name a defined VLAN.";
}
{
assertion = lib.all (p: !(cfg.accessPorts ? ${p})) cfg.trunkPorts;
message = "cnx.router: a port cannot be both a trunk and an access port.";
}
];
# Router diagnostics toolkit: packets (tcpdump), path (mtr), link
# negotiation (ethtool), NAT state (conntrack), DNS (kdig), per-flow
# bandwidth (iftop), WAN throughput (librespeed-cli; iperf3 covers LAN).
environment.systemPackages = with pkgs; [
tcpdump
mtr
ethtool
conntrack-tools
knot-dns
iftop
librespeed-cli
];
networking.useNetworkd = true;
@@ -235,6 +291,21 @@ in
};
}) cfg.trunkPorts
)
// lib.mapAttrs' (
port: vlanName:
lib.nameValuePair "25-access-${port}" {
matchConfig.Name = port;
networkConfig.Bridge = "br0";
bridgeVLANs = [
{
VLAN = cfg.vlans.${vlanName}.id;
PVID = cfg.vlans.${vlanName}.id;
EgressUntagged = cfg.vlans.${vlanName}.id;
}
];
linkConfig.RequiredForOnline = "no";
}
) cfg.accessPorts
// lib.mapAttrs' (
name: vlan:
lib.nameValuePair "40-${vlanIf name}" {
+5
View File
@@ -29,6 +29,11 @@ in
interface = "vlan-${name}";
valid-lifetime = vlan.dhcp.leaseTime;
pools = [ { pool = "${vlan.dhcp.pool.from} - ${vlan.dhcp.pool.to}"; } ];
reservations = lib.mapAttrsToList (host: res: {
hostname = host;
hw-address = res.hwAddress;
ip-address = res.ipAddress;
}) vlan.dhcp.reservations;
option-data = [
{
name = "routers";
+28
View File
@@ -0,0 +1,28 @@
# iperf3 server on every gateway, for throughput testing from any LAN segment
# (e.g. validating AP/switch links: `iperf3 -c 10.<siteId>.<vlan>.1`) and from
# admin machines over the mesh. Never reachable from the WAN (default-deny).
{
config,
lib,
...
}:
let
cfg = config.cnx.router;
mesh = import ../mesh-hosts.nix { inherit config lib; };
vlanIfs = lib.mapAttrsToList (name: _: "vlan-${name}") cfg.vlans;
in
{
config = lib.mkIf cfg.enable {
services.iperf3.enable = true;
networking.firewall.interfaces = lib.genAttrs vlanIfs (_: {
allowedTCPPorts = [ 5201 ];
allowedUDPPorts = [ 5201 ];
});
networking.firewall.extraInputRules = ''
ip6 saddr ${mesh.subnet} tcp dport 5201 accept comment "iperf3 over the mesh"
ip6 saddr ${mesh.subnet} udp dport 5201 accept comment "iperf3 over the mesh"
'';
};
}
+67
View File
@@ -0,0 +1,67 @@
# Periodic WAN speed test so ISP degradation shows up as a trend instead of a
# complaint. A timer runs librespeed-cli and writes the results as Prometheus
# metrics into node_exporter's textfile collector — they ride the existing
# 9100 scrape to VictoriaMetrics, where alerts.nix compares each run against
# the link's own 7-day median (no per-site threshold to maintain).
{
config,
lib,
pkgs,
...
}:
let
cfg = config.cnx.router;
textfileDir = "/var/lib/speedtest";
in
{
options.cnx.router.speedtest.interval = lib.mkOption {
type = lib.types.str;
default = "hourly";
description = "systemd OnCalendar spec for the WAN speed test.";
};
config = lib.mkIf cfg.enable {
services.prometheus.exporters.node.extraFlags = [
"--collector.textfile.directory=${textfileDir}"
];
systemd.services.speedtest = {
description = "WAN speed test to Prometheus textfile metrics";
after = [ "network-online.target" ];
wants = [ "network-online.target" ];
path = [
pkgs.librespeed-cli
pkgs.jq
];
serviceConfig = {
Type = "oneshot";
StateDirectory = "speedtest";
# One test at boot would race PPPoE and log a spurious failure.
ExecCondition = "${pkgs.iproute2}/bin/ip link show ppp0";
};
script = ''
tmp="${textfileDir}/.speedtest.prom.tmp"
if result=$(librespeed-cli --json); then
jq -r '.[0]
| "speedtest_download_mbps \(.download)",
"speedtest_upload_mbps \(.upload)",
"speedtest_ping_ms \(.ping)",
"speedtest_jitter_ms \(.jitter)",
"speedtest_success 1"' <<<"$result" > "$tmp"
else
echo "speedtest_success 0" > "$tmp"
fi
mv "$tmp" "${textfileDir}/speedtest.prom"
'';
};
systemd.timers.speedtest = {
wantedBy = [ "timers.target" ];
timerConfig = {
OnCalendar = cfg.speedtest.interval;
RandomizedDelaySec = "10m";
Persistent = true;
};
};
};
}