feat(collector): forecast and ntfy in ping-collector; ThaiWater keeps alerts current at night
CI / Format & lint (push) Successful in 22s
CI / Test suite (push) Successful in 59s
Docs / Validate documentation (push) Successful in 25s
Rust / fmt, clippy, test, build (push) Successful in 10m2s
Security / Dependency vulnerabilities (push) Successful in 2m11s
Security / Static analysis (push) Successful in 22s
Security / License report (push) Successful in 1m37s

Phase 2b/2c of docs/RUST_REWRITE.md. With FORECAST_IN_COLLECTOR=true (shared
.env) ping-collector, after each cycle that brought newer RID or HII readings
and at least hourly: loads the last 336 h (RID + HII mirror fill), fetches the
rain forecast, runs every exported model (rust/forecast), stores
forecast_history and forecast_current, and sends the ntfy notifications.
The Python API then serves forecast_current as /forecast (computing in-process
only when it is older than FORECAST_MAX_AGE_SECONDS) and neither forecasts nor
notifies.

ntfy (collector/src/notify.rs): a port of src/notify.py with the same topics,
texts, hysteresis, capacity guard and notification_state keys, checked word
for word on 751 messages generated by Python (testdata/gen_notify_cases.py).
The change: for stations whose ThaiWater twin is the same sensor, a reading
newer than RID's drives the level alerts (marked as ThaiWater telemetry), and
the feed counts as stale only when neither source has a reading for 3 h. RID
alone lagging is reported once after RID_STALE_ALERT_HOURS (18). This ends the
nightly "gauge feed stale" / "recovered" pairs caused by RID's late reports.

Commands: `forecast [--json] [--save] [--now] [--rain-file]`,
`notify --dry-run`. scripts/forecast_parity.py compares Python and the
collector on the live database (same moment, same rain); locally 48/48 rows
identical. install_collector.sh: models export + that check on every install,
--cutover-forecast (refuses on a failed check) and --rollback-forecast.

Trees are packed into 24-byte nodes: 70 MB peak with all 15 models (was 140).
Forecast values are written to NUMERIC as shortest round-trip text, so they
round as psycopg2's do. DB test schemas get a counter (parallel tests collided
on Windows' coarse clock).
This commit is contained in:
grabowski committed 2026-10-18 14:43:39 +07:00
1 parent 5c4f464383
commit 6ae67dec63
27 files changed
+2656 -118

No files matched your search

+11 -1
View File
@@ -33,6 +33,14 @@ Production: one systemd unit on a small VPS, `/opt/thailand-water-monitor`, user
collector writes the same tables as the Python collectors: a change to a table,
upsert or parser on one side must be made on the other until the Python side is
retired (check with a live run of both into two databases).
- The forecast is served by Rust (`rust/forecast`) from models trained in Python.
A change to `src/ml/features.py`, `predict.py`, the HII fill or the dam frame
must be ported in the same change: regenerate
`rust/forecast/testdata/gen_forecast_fixtures.py` and keep
`cargo test -p ping-forecast` green. Changing thresholds, stages, leads or
notify constants: `uv run python -m src.ml.export --config` (a test fails
otherwise) and rebuild the collector. Notify texts:
`rust/collector/testdata/gen_notify_cases.py`.
## Where things are
@@ -42,6 +50,8 @@ Production: one systemd unit on a small VPS, `/opt/thailand-water-monitor`, user
- `scripts/retrain.sh` + `water-monitor-retrain.timer`: monthly retrain with staged
promote. `scripts/dev_proxy.py`: serve the working-copy dashboard against the live API.
- `rust/collector` (`ping-collector`): Rust data collector; with it deployed the
Python app runs `COLLECTION_ENABLED=false`. Rewrite plan: `docs/RUST_REWRITE.md`.
Python app runs `COLLECTION_ENABLED=false`. With `FORECAST_IN_COLLECTOR=true` it
also forecasts (`rust/forecast`) and sends ntfy; the API serves its
`forecast_current`. Rewrite plan: `docs/RUST_REWRITE.md`.
- `docs/FLOOD_FORECASTING.md` is the authoritative model write-up; `docs/DATA_SOURCES.md`
the source catalog.