feat(collector): forecast and ntfy in ping-collector; ThaiWater keeps alerts current at night
CI / Format & lint (push) Successful in 22s
CI / Test suite (push) Successful in 59s
Docs / Validate documentation (push) Successful in 25s
Rust / fmt, clippy, test, build (push) Successful in 10m2s
Security / Dependency vulnerabilities (push) Successful in 2m11s
Security / Static analysis (push) Successful in 22s
Security / License report (push) Successful in 1m37s

Phase 2b/2c of docs/RUST_REWRITE.md. With FORECAST_IN_COLLECTOR=true (shared
.env) ping-collector, after each cycle that brought newer RID or HII readings
and at least hourly: loads the last 336 h (RID + HII mirror fill), fetches the
rain forecast, runs every exported model (rust/forecast), stores
forecast_history and forecast_current, and sends the ntfy notifications.
The Python API then serves forecast_current as /forecast (computing in-process
only when it is older than FORECAST_MAX_AGE_SECONDS) and neither forecasts nor
notifies.

ntfy (collector/src/notify.rs): a port of src/notify.py with the same topics,
texts, hysteresis, capacity guard and notification_state keys, checked word
for word on 751 messages generated by Python (testdata/gen_notify_cases.py).
The change: for stations whose ThaiWater twin is the same sensor, a reading
newer than RID's drives the level alerts (marked as ThaiWater telemetry), and
the feed counts as stale only when neither source has a reading for 3 h. RID
alone lagging is reported once after RID_STALE_ALERT_HOURS (18). This ends the
nightly "gauge feed stale" / "recovered" pairs caused by RID's late reports.

Commands: `forecast [--json] [--save] [--now] [--rain-file]`,
`notify --dry-run`. scripts/forecast_parity.py compares Python and the
collector on the live database (same moment, same rain); locally 48/48 rows
identical. install_collector.sh: models export + that check on every install,
--cutover-forecast (refuses on a failed check) and --rollback-forecast.

Trees are packed into 24-byte nodes: 70 MB peak with all 15 models (was 140).
Forecast values are written to NUMERIC as shortest round-trip text, so they
round as psycopg2's do. DB test schemas get a counter (parallel tests collided
on Windows' coarse clock).
This commit is contained in:
grabowski committed 2026-10-18 14:43:39 +07:00
1 parent 5c4f464383
commit 6ae67dec63
27 files changed
+2656 -118

No files matched your search

+10
View File
@@ -62,6 +62,13 @@ GAP_FILL_DAYS_ON_START=14
# share the status file between both services for /scraping/status
COLLECTION_ENABLED=true
# SCRAPING_STATUS_FILE=/opt/thailand-water-monitor/collector-status.json
# true: ping-collector also forecasts (models/flood_*.json.gz, rust/forecast)
# and sends the ntfy notifications; this app serves its stored forecast and
# computes itself only when that is older than FORECAST_MAX_AGE_SECONDS.
# Set by scripts/install_collector.sh --cutover-forecast.
FORECAST_IN_COLLECTOR=false
# FORECAST_MAX_AGE_SECONDS=7200
# MODELS_DIR=models
REQUEST_TIMEOUT=30
MAX_RETRIES=3
RETRY_DELAY_SECONDS=60
@@ -105,6 +112,9 @@ NTFY_PUBLISH_URL=
NTFY_TOPIC_PREFIX=ping
NTFY_TOKEN=
PUBLIC_URL=https://water.buildfor.life/
# ping-collector notifier: report RID lagging this many hours while ThaiWater
# telemetry is current (0 = never)
RID_STALE_ALERT_HOURS=18
# Matrix Alerting Configuration
MATRIX_HOMESERVER=https://matrix.org