grabowski
764764e07e
train_all() now raises RainUnavailableError when use_rain=True and the
Open-Meteo history cannot be loaded, instead of logging a warning and
writing gauge-only (v2) bundles over the deployed v3 set -- which is what
the 2026-09-01 server retrain did unnoticed. --no-rain remains the explicit
way to get v2. CLI exits 2 with a one-line error. Three tests cover the
guard, the opt-out, and the v3 happy path.
scripts/retrain.sh trains into models/.staging, refuses to promote unless
metrics.json shows hgb-v3+ and >=14 trained stations, then renames bundles
into place (previous generation kept in models/.previous). No API restart:
predict.py reloads by mtime on the hourly precompute.
water-monitor-retrain.{service,timer}: 1st of each month 03:30, Persistent,
OMP_NUM_THREADS=4, Nice=15, same sandbox as the API unit. install.sh now
does `uv sync` into .venv (one env rule; removes a stale venv/) and enables
the timer. water-monitor.service in the repo matched neither the deployed
unit nor the uv env; it now does (run.py --web-api, .venv, EnvironmentFile).
42 lines
1.2 KiB
Desktop File
42 lines
1.2 KiB
Desktop File
[Unit]
|
|
Description=Thailand Water Level Monitor
|
|
Documentation=https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor
|
|
After=network.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=water-monitor
|
|
Group=water-monitor
|
|
WorkingDirectory=/opt/thailand-water-monitor
|
|
# The uv-managed env (uv sync -> .venv). Same interpreter for water-monitor-retrain.service.
|
|
ExecStart=/opt/thailand-water-monitor/.venv/bin/python run.py --web-api
|
|
ExecReload=/bin/kill -HUP $MAINPID
|
|
Restart=always
|
|
RestartSec=60
|
|
TimeoutStopSec=30
|
|
|
|
# DB_TYPE / POSTGRES_CONNECTION_STRING / MATRIX_* come from the .env file.
|
|
EnvironmentFile=/opt/thailand-water-monitor/.env
|
|
Environment=PYTHONPATH=/opt/thailand-water-monitor
|
|
# Serving path is latency-bound; single-threaded BLAS is 2.6x faster per call
|
|
# (docs/FLOOD_FORECASTING.md section 6). Training sets its own value.
|
|
Environment=OMP_NUM_THREADS=1
|
|
Environment=PYTHONUNBUFFERED=1
|
|
|
|
# Security settings
|
|
NoNewPrivileges=true
|
|
PrivateTmp=true
|
|
ProtectSystem=strict
|
|
ProtectHome=true
|
|
ReadWritePaths=/opt/thailand-water-monitor
|
|
CapabilityBoundingSet=
|
|
|
|
# Logging
|
|
StandardOutput=journal
|
|
StandardError=journal
|
|
SyslogIdentifier=water-monitor
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|