feat(forecast): Rust forecast crate, bit-for-bit with the Python serving path
CI / Test suite (push) Successful in 1m2s
CI / Format & lint (push) Successful in 22s
Rust / fmt, clippy, test, build (push) Successful in 9m53s
Security / Dependency vulnerabilities (push) Successful in 1m52s
Security / Static analysis (push) Successful in 20s
Security / License report (push) Successful in 1m41s
CI / Test suite (push) Successful in 1m2s
CI / Format & lint (push) Successful in 22s
Rust / fmt, clippy, test, build (push) Successful in 9m53s
Security / Dependency vulnerabilities (push) Successful in 1m52s
Security / Static analysis (push) Successful in 20s
Security / License report (push) Successful in 1m41s
Phase 2a of docs/RUST_REWRITE.md. rust/forecast (ping-forecast) ports the serving half of src/ml: the HII mirror gap fill, the hourly grid, every feature (pandas' Kahan rolling sums included), the Mae Ngat dam frame, HistGradientBoosting tree inference, the heuristic fallback, stale-threshold handling and the P.1 flood-stage probabilities. Hand-over from Python (src/ml/export.py): - models/flood_<station>.json.gz next to each joblib bundle; train_all() writes it, retrain.sh promotes it, `python -m src.ml.export` exports existing ones. - rust/forecast/flood_config.json: thresholds, stages, leads and notify constants, compiled in. tests/test_model_export.py fails while it differs from the Python source. Parity (rust/forecast/tests/parity.rs, fixtures from testdata/gen_forecast_fixtures.py on the production code path): four scenarios incl. the 2024 flood with RID lagging behind HII. Merged readings, all features and all levels identical; with all 15 local models 377/384 probabilities bit-identical, the rest within 2 ulp (exp). Python refactor without behaviour change: predict.forecasts_from_long, and get_latest_forecasts/load_latest/serving_frame take now/rain for reproducible comparisons; the HII query orders its rows.
This commit is contained in:
1 parent
6d4beae65a
commit
5c4f464383
31 files changed
+2889
-29
No files matched your search
+5
-4
@@ -11,9 +11,9 @@
|
||||
# * A run that produced gauge-only (v2) bundles, or trained too few stations,
|
||||
# must NOT replace the deployed models. train.py already aborts on a
|
||||
# missing rain series; this script re-checks the written metrics anyway.
|
||||
# * No API restart is needed: predict.py reloads changed bundles on the next
|
||||
# precompute (every scrape cycle, hourly), so the new models are live
|
||||
# within an hour. Restart manually if you want them live immediately.
|
||||
# * No restart is needed: predict.py (Python) and ping-collector (Rust,
|
||||
# FORECAST_IN_COLLECTOR=true) reload changed bundles on their next
|
||||
# forecast run, so the new models are live within an hour.
|
||||
#
|
||||
# Exit codes: 0 ok, 2 training refused (see log), 3 verification failed.
|
||||
set -euo pipefail
|
||||
@@ -79,7 +79,8 @@ fi
|
||||
# Promote: per-file rename is atomic; readers see either the old or the new
|
||||
# bundle, never a partial one. Keep one previous generation for rollback.
|
||||
mkdir -p "${MODELS_DIR}/.previous"
|
||||
for f in "${STAGE_DIR}"/flood_*.joblib "${STAGE_DIR}/metrics.json"; do
|
||||
# flood_*.json.gz is the Rust forecaster's export of the same bundle.
|
||||
for f in "${STAGE_DIR}"/flood_*.joblib "${STAGE_DIR}"/flood_*.json.gz "${STAGE_DIR}/metrics.json"; do
|
||||
name="$(basename "${f}")"
|
||||
if [ -f "${MODELS_DIR}/${name}" ]; then
|
||||
mv -f "${MODELS_DIR}/${name}" "${MODELS_DIR}/.previous/${name}"
|
||||
|
||||
Reference in new issue
Block a user