feat(collector): forecast and ntfy in ping-collector; ThaiWater keeps alerts current at night
CI / Format & lint (push) Successful in 22s
CI / Test suite (push) Successful in 59s
Docs / Validate documentation (push) Successful in 25s
Rust / fmt, clippy, test, build (push) Successful in 10m2s
Security / Dependency vulnerabilities (push) Successful in 2m11s
Security / Static analysis (push) Successful in 22s
Security / License report (push) Successful in 1m37s

Phase 2b/2c of docs/RUST_REWRITE.md. With FORECAST_IN_COLLECTOR=true (shared
.env) ping-collector, after each cycle that brought newer RID or HII readings
and at least hourly: loads the last 336 h (RID + HII mirror fill), fetches the
rain forecast, runs every exported model (rust/forecast), stores
forecast_history and forecast_current, and sends the ntfy notifications.
The Python API then serves forecast_current as /forecast (computing in-process
only when it is older than FORECAST_MAX_AGE_SECONDS) and neither forecasts nor
notifies.

ntfy (collector/src/notify.rs): a port of src/notify.py with the same topics,
texts, hysteresis, capacity guard and notification_state keys, checked word
for word on 751 messages generated by Python (testdata/gen_notify_cases.py).
The change: for stations whose ThaiWater twin is the same sensor, a reading
newer than RID's drives the level alerts (marked as ThaiWater telemetry), and
the feed counts as stale only when neither source has a reading for 3 h. RID
alone lagging is reported once after RID_STALE_ALERT_HOURS (18). This ends the
nightly "gauge feed stale" / "recovered" pairs caused by RID's late reports.

Commands: `forecast [--json] [--save] [--now] [--rain-file]`,
`notify --dry-run`. scripts/forecast_parity.py compares Python and the
collector on the live database (same moment, same rain); locally 48/48 rows
identical. install_collector.sh: models export + that check on every install,
--cutover-forecast (refuses on a failed check) and --rollback-forecast.

Trees are packed into 24-byte nodes: 70 MB peak with all 15 models (was 140).
Forecast values are written to NUMERIC as shortest round-trip text, so they
round as psycopg2's do. DB test schemas get a counter (parallel tests collided
on Windows' coarse clock).
This commit is contained in:
grabowski committed 2026-10-18 14:43:39 +07:00
1 parent 5c4f464383
commit 6ae67dec63
27 files changed
+2656 -118

No files matched your search

+17 -10
View File
@@ -1,6 +1,7 @@
# A Rust rewrite: assessment
Status: **in progress.** Phase 1 (collector) is in production since 2026-10-17; see
Status: **in progress.** Phase 1 (collector) is in production since 2026-10-17;
phase 2 (forecast + ntfy) is built and parity-verified, cutover pending; see
[rust/README.md](../rust/README.md). Spike: [spikes/rust-forecast](../spikes/rust-forecast/README.md).
## Short version
@@ -51,10 +52,10 @@ One binary, one process (tokio):
the 257 KB river GeoJSON), static dashboard, CORS; `utoipa` if `/docs` stays.
- sqlx against the existing schema. Python and Rust can run against the same
database during migration.
- models: `scripts/retrain.sh` also exports each bundle to JSON (spike format,
~4.9 MB per station vs 6.2 MB joblib). Rust loads them at start and reloads on
change. A bundle whose `feature_names` Rust cannot build is refused and the
station falls back to the heuristic, as Python does today.
- models: `train_all()` also exports each bundle (`models/flood_<station>.json.gz`,
1.2-2.3 MB), `scripts/retrain.sh` promotes it. Rust reloads changed files on
its next run. A bundle whose `feature_names` Rust cannot build falls back to
the heuristic, as Python does.
## The permanent cost: the Python/Rust model contract
@@ -83,7 +84,7 @@ This slows model iteration, which is the most active part of the repo.
| --- | --- | --- |
| 0 | Spike: P.1 features + HGB inference | done, bit-identical |
| 1 | `ping-collector`: RID, HII, dams, Open-Meteo | in production since 2026-10-17 23:09 (Python: `COLLECTION_ENABLED=false`) |
| 2 | Forecast crate + ntfy | next |
| 2 | Forecast crate + ntfy | built 2026-10-18, parity-verified; cutover with `install_collector.sh --cutover-forecast` |
| 3 | Read API | |
| 4 | Cleanup | |
@@ -95,10 +96,16 @@ This slows model iteration, which is the most active part of the repo.
RID reading in the database advances. ntfy moved to phase 2: the outlook
alert needs the forecasts, which are still Python. Cutover and rollback
are one command each (`scripts/install_collector.sh`).
2. **Forecast crate + ntfy** (~2k LOC): all 15 stations, dam/HII/rain inputs,
heuristic fallback, probabilities and stages, `forecast_history`,
`notification_state`; parity CI for every station on a flood window and a
dry window. The Python API then only serves.
2. **Forecast crate + ntfy** (`rust/forecast`, run by `ping-collector` with
`FORECAST_IN_COLLECTOR=true`): all stations, HII/rain/dam inputs, heuristic
fallback, probabilities and stages, `forecast_history`, `forecast_current`
(served by the API) and `notification_state`. Parity: fixtures from the
Python production path (CI: P.1 model; locally all 15), a live comparison
script, and 751 notifications replayed word for word. Porting found a
Python serving bug, fixed first: forecast rain (`rain_fc24`) was always
NaN at serving time (docs/FLOOD_FORECASTING.md section 5). The ntfy change:
ThaiWater mirror readings keep level alerts and the feed check current
while RID lags at night. The Python API then only serves.
3. **Read API** (~2k LOC): routes recorded from the Python API as golden files,
then a Caddy switch. Python keeps only training/evaluation.
4. **Cleanup:** drop the multi-backend adapters, demo databases and the