Add VictoriaLogs for centralized journald across all hosts
control runs VictoriaLogs (:9428, 30d, mesh-scoped) with a matching Grafana datasource. Each host ships journald via systemd's own journald.upload to the /insert/journald endpoint -- no extra agent. control uploads over loopback so its logs survive a mesh outage; ns1 and ns2 push over the mesh.
This commit is contained in:
+20
-1
@@ -42,6 +42,25 @@ Dashboards are provisioned from `modules/monitoring/dashboards/` (any JSON file
|
||||
there is picked up):
|
||||
|
||||
- **CNX DNS** (`dns.json`) — firing alerts, per-nameserver SOA serials, zone
|
||||
expiry countdowns, query/response rates, and host CPU/memory/disk/load.
|
||||
expiry countdowns, query/response rates, host CPU/memory/disk/load, and the
|
||||
outside-in DNS probes.
|
||||
- **CNX Backups** (`backups.json`) — borgbackup job health, time since the last
|
||||
run, and per-job state. See [Backups](./backups.md).
|
||||
|
||||
## Logs
|
||||
|
||||
**VictoriaLogs** on `control` (`:9428`), 30-day retention
|
||||
(`modules/monitoring/server.nix`). All three hosts ship journald to it via
|
||||
systemd's own `services.journald.upload` → the `/insert/journald` endpoint
|
||||
(`modules/monitoring/exporters.nix`); no extra agent. `control` uploads over
|
||||
loopback so its logs survive a mesh outage, `ns1`/`ns2` push over the mesh, and
|
||||
9428 is firewall-scoped to the mesh like everything else.
|
||||
|
||||
Query logs from Grafana via the provisioned **VictoriaLogs** datasource (Explore
|
||||
view, LogsQL), or directly in the built-in UI at `http://[control]:9428/select/vmui`.
|
||||
Logs are tagged with `_HOSTNAME` and `_SYSTEMD_UNIT`, so to follow one service
|
||||
across hosts:
|
||||
|
||||
```
|
||||
_SYSTEMD_UNIT:"knot.service"
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user