feat(forecast): Rust forecast crate, bit-for-bit with the Python serving path
CI / Test suite (push) Successful in 1m2s
CI / Format & lint (push) Successful in 22s
Rust / fmt, clippy, test, build (push) Successful in 9m53s
Security / Dependency vulnerabilities (push) Successful in 1m52s
Security / Static analysis (push) Successful in 20s
Security / License report (push) Successful in 1m41s

Phase 2a of docs/RUST_REWRITE.md. rust/forecast (ping-forecast) ports the
serving half of src/ml: the HII mirror gap fill, the hourly grid, every
feature (pandas' Kahan rolling sums included), the Mae Ngat dam frame,
HistGradientBoosting tree inference, the heuristic fallback, stale-threshold
handling and the P.1 flood-stage probabilities.

Hand-over from Python (src/ml/export.py):
- models/flood_<station>.json.gz next to each joblib bundle; train_all() writes
  it, retrain.sh promotes it, `python -m src.ml.export` exports existing ones.
- rust/forecast/flood_config.json: thresholds, stages, leads and notify
  constants, compiled in. tests/test_model_export.py fails while it differs
  from the Python source.

Parity (rust/forecast/tests/parity.rs, fixtures from
testdata/gen_forecast_fixtures.py on the production code path): four
scenarios incl. the 2024 flood with RID lagging behind HII. Merged readings,
all features and all levels identical; with all 15 local models 377/384
probabilities bit-identical, the rest within 2 ulp (exp).

Python refactor without behaviour change: predict.forecasts_from_long, and
get_latest_forecasts/load_latest/serving_frame take now/rain for
reproducible comparisons; the HII query orders its rows.
This commit is contained in:
grabowski committed 2026-10-18 14:21:56 +07:00
1 parent 6d4beae65a
commit 5c4f464383
31 files changed
+2889 -29

No files matched your search

+5 -4
View File
@@ -11,9 +11,9 @@
# * A run that produced gauge-only (v2) bundles, or trained too few stations,
# must NOT replace the deployed models. train.py already aborts on a
# missing rain series; this script re-checks the written metrics anyway.
# * No API restart is needed: predict.py reloads changed bundles on the next
# precompute (every scrape cycle, hourly), so the new models are live
# within an hour. Restart manually if you want them live immediately.
# * No restart is needed: predict.py (Python) and ping-collector (Rust,
# FORECAST_IN_COLLECTOR=true) reload changed bundles on their next
# forecast run, so the new models are live within an hour.
#
# Exit codes: 0 ok, 2 training refused (see log), 3 verification failed.
set -euo pipefail
@@ -79,7 +79,8 @@ fi
# Promote: per-file rename is atomic; readers see either the old or the new
# bundle, never a partial one. Keep one previous generation for rollback.
mkdir -p "${MODELS_DIR}/.previous"
for f in "${STAGE_DIR}"/flood_*.joblib "${STAGE_DIR}/metrics.json"; do
# flood_*.json.gz is the Rust forecaster's export of the same bundle.
for f in "${STAGE_DIR}"/flood_*.joblib "${STAGE_DIR}"/flood_*.json.gz "${STAGE_DIR}/metrics.json"; do
name="$(basename "${f}")"
if [ -f "${MODELS_DIR}/${name}" ]; then
mv -f "${MODELS_DIR}/${name}" "${MODELS_DIR}/.previous/${name}"