5066d4ee46515d38ce4955553f3eeef97b2ea0b8
mirach and almach have each hung at least twice since July with nothing in the journal or pstore, and stayed down until someone power-cycled the board. RuntimeWatchdogSec was 0 and kernel.panic was 0, so a hung kernel/PID1 or a panic could never self-recover. Arm the systemd hardware watchdog (30s) and reboot 10s after a panic on all phonebox nodes. Watchdog resets will show up as extra entries in journalctl --list-boots.
infra
Infrastructure for NewEdge
Languages
Nix
100%