8 Commits

Include renames

Author	SHA1	Message	Date
grabowski	6d4beae65a	fix(ml): serve the forecast rain the model was trained with ... build_features cut the rain series to the gauge grid before the forward sum, and the grid ends at the newest reading, so the row every live forecast is made from had rain_fc24 = NaN. Training and all evaluations had the real Open-Meteo forecast there. The forward sum now runs over the grid extended by 24 h; earlier rows are unchanged. Harness (new opt-in variant rise_rain_fc24_missing = rise_rain with rain_fc24 blanked on test rows, five folds, --from-cache): P.1 and P.103 nearly unaffected, P.21 false alarms 10 -> 5 and MAE 0.232 -> 0.215 with the fix. Write-up in docs/FLOOD_FORECASTING.md section 5; results in models/eval_2026-10-18_fc24.json. Also: HII mirror rows in one hour now resolve to the latest reading via a stable sort on the raw time (was an unstable sort on the floored hour, i.e. row-order dependent). Needed for an exact Rust port.	2026-10-18 14:07:56 +07:00
grabowski	32399f1899	feat: recalibrate P.77/P.75 thresholds; capacity guard on alerts ... The first ntfy cycle announced "Warning level at P.77" at 3.02 m. That gauge's 2.85 m threshold sat below its own dry-season baseline (2.6-2.7 m at 8-14 % channel capacity): P.77 had been "above warning" for 761 of the last 2 146 hours, at 22 % capacity. Across 2018-2024, 75-85 % capacity reads 3.35-4.57 m and 95-105 % reads 4.27-5.08 m; set 4.30 / 4.90. P.75 moved 2.75/3.50 -> 3.20/3.65 on the same evidence (2024: 3.45 / 3.72). The predictor already handles changed thresholds (regression-derived probabilities until the Oct 1 retrain). Second line of defence in notify.py: a clear->alert transition is only announced when RID's discharge_percent for the reading is >= 60 %, so a re-rated or datum-shifted gauge cannot page subscribers again. P.1 is exempt (its stages come from the inundation map, not capacity); readings without a capacity figure fall back to level only; the all-clear edge is never blocked. 3 tests.	2026-09-12 00:34:59 +02:00
grabowski	28b62e5a36	feat: Mae Ngat dam features — built, evaluated, defaulted OFF ... src/ml/dam.py loads rid_reservoir_daily into a leakage-safe hourly frame (daily row visible from 07:00 its own date, ffill capped at 48 h) and is plumbed through features/train/predict/evaluate exactly like rain, gated to the six mainstem stations below the Mae Ngat confluence. The experiment concludes as a documented NEGATIVE result: on the 2024 record-flood backtest every dam-feature subset costs 1-3 h of first-alert lead (13h -> 10-12h) for <=3 cm of peak-error gain, because the daily RID report lags up to 31 h and describes yesterday's benign absorbing reservoir during fast onset. Features therefore default OFF (--dam opt-in on the training and backtest CLIs; rise_rain_dam/rise_dam harness variants, excluded from the default variant set). The ablation also isolated the HII gap-fill as lead-neutral: the acceptance gate holds at 13 h with fill enabled, and docs/img charts are regenerated with the shipping configuration. Full table in docs/FLOOD_FORECASTING.md §5. Review-swarm fixes: evaluate.py skips variants whose feature family is absent instead of crashing the run; --dam forwards --db-url and warns loudly when no dam history loads; an empty DB result can no longer wipe a good dam cache; run-level metrics version claims v4 only when a dam station is actually in the set.	2026-08-13 20:42:21 +07:00
grabowski	cbb3bf7369	feat: Open-Meteo catchment rainfall series + rain features + harness variant ... src/ml/rain.py fetches hourly precipitation for five upper-Ping catchment points (Chiang Dao, Mae Taeng, Mae Ngat, Mae Rim, city) from the Open-Meteo forecast-model archive (2021-03 onward, no API key, Bangkok-local timestamps, year-chunked local cache) plus the live forecast endpoint for serving (trailing days + next 48h). features.build_features/build_matrix accept the catchment-mean series and add rain_6h/24h/72h trailing sums and rain_fc24 — the forward 24h sum, a genuine forecast feature (archived forecasts at training time, a real weather forecast at serving; never contains river data). Columns exist only when a series is provided: HGB rejects all-NaN columns at fit, so no-rain training omits them and serving passes an empty series for alignment. evaluate.py gains the rise_rain variant (and --no-rain) so the harness can judge whether rain beats the deployed rise baseline.	2026-08-12 16:39:36 +07:00
grabowski	a0086086a2	feat: rolling-origin event-aware evaluation harness for model variants ... One fold per monsoon season (train <= 30 Apr, test Jun-Nov, 2021-2025) replaces the single fixed holdout that contained only ~4 warning events. Metrics are what matters operationally: sustained first-alert lead vs each observed 3.70m crossing (two consecutive alerting samples required; lookback floored at the previous event's end so multi-peak floods can't launder lead credit), peak error from the prediction actually issued 24h before the peak (3h match tolerance, null on outages), false-alarm episodes (12h gap tolerance), MAE / flood-regime MAE, and a Brier score on warning exceedance — included because sigma cancels algebraically in any p>=0.5 alert metric, so lead times compare predictors while Brier compares uncertainty models. Variants: baseline_abs (current), rise (target = future max - current level), rise_weighted (flood-regime sample weights 1x->5x), and rise_quantile (q50/q90 heads, spread-implied sigma). Harness verified by a 3-agent adversarial review (features bit-identical across fold cutoffs; three metric flaws found and fixed before first use). Also: features.build_labels/build_matrix gain stats_end so the rescue quantile is computed from pre-cutoff data only, closing the label- construction leak flagged in the earlier ML review.	2026-08-12 15:19:26 +07:00
grabowski	ecd34177bb	fix: backtest/review findings in the flood-ML package ... From the adversarial review and threshold backtest (swarm verification): - predict.py: when a bundle's trained thresholds differ from the current config (deploy before retrain), skip its stale classifier heads and derive p_warning/p_danger from the regression + sigma against the CURRENT thresholds - the dashboard can no longer show contradictory old-threshold classifier output next to new-threshold stages - features.py: decouple the low-coverage regression-label rescue from the warning threshold (now the station's own p97.5 level); the old coupling silently dropped 34% of P.5's regression training rows and cost +46% MAE when its threshold rose - features.py: P.82 danger 3.80 -> 3.75 (3.80 was above the station's 8-year maximum of 3.78, so danger could never train or fire) - data.py / predict.py: anchor models/cache paths to the repo root; the relative paths silently returned zero rows when run from another CWD - annotate P.4A thresholds as low-confidence (11 supporting readings) 47 tests pass. Retrain required for the label-rescue and P.82 changes to reach the classifier heads.	2026-08-10 15:57:01 +07:00
grabowski	e4d5d274f0	feat: per-station flood thresholds and Chiang Mai inundation stages for P.1 ... Replace the network-wide (3.0, 4.5) m thresholds with per-station values calibrated from the DB's discharge_percent (RID % of channel capacity): warning = median level at 75-85% capacity, danger = median at 95-105%. Fixes P.103 over-alerting (bank-full ~6.75 m, not 4.5) and P.67 under-alerting (overflow ~2.9 m). Requires a retrain to take effect in the classifier heads. P.1 uses the official Chiang Mai municipal inundation map instead: warning 3.70 m (stage 1, city flooding begins), danger 4.20 m (stage 5), with the full 7-stage table (3.70-4.60 m + discharge) in features.P1_FLOOD_STAGES. Forecast rows for P.1 now include per-stage exceedance probabilities computed from the regression head + calibration sigma - available immediately without retraining. Dashboard: "Chiang Mai city flood outlook" block above the forecast grid (predicted peak + 7 stage-probability chips) and a toggleable georeferenced overlay of the official flood-zone map (static/flood-zones-p1.jpg, bounds tunable in FLOOD_ZONE_BOUNDS).	2026-08-10 15:35:00 +07:00
grabowski	4358d52d55	feat: ML flood-event forecasting from 8 years of gauge history ... Add src/ml/ package predicting, per station and per 6/12/24 h horizon, the probability of exceeding warning (3.0 m) and danger (4.5 m) levels plus expected peak level, trained on the 592k-row PostgreSQL history: - features.py: hourly grid with coverage gating and no future leakage; upstream stations enter at empirically measured travel-time lags (P.20 +17h ... P.103 +1h vs P.1); hour-of-day deliberately excluded (it encodes the scrape schedule, not hydrology) - train.py: HistGradientBoosting regression + warn/danger classifier heads per station x horizon, >=30-positives gate with calibrated sigmoid-on-regression fallback, strict temporal splits, per-event lead-time evaluation; guards against sklearn 1.9.0 crash on degenerate feature columns - predict.py: bundle loading with feature-name checks, heuristic fallback tier, get_latest_forecasts() for the API; raises when no models are trained so the endpoint 503s instead of serving persistence output as forecasts - data.py: Postgres-first loader (FLOOD_ML_DB_URL override), HTTP API fallback (flagged: that path backfills synthetic discharge), csv.gz cache - /forecast endpoint (15-min TTL cache) + dashboard flood-risk panel (hidden until models exist) - docs/FLOOD_FORECASTING.md: full system doc with measured deployment numbers (~335 MB RSS, CPU negligible, ~6 min full retrain) and retraining policy Validation: out-of-sample backtest of the record 2024 flood season (train <= Aug 2024) alerted 24-48 h ahead of the Oct 5 peak; 2025-26 test split: P.1 6h PR-AUC 0.974, recall 98.3% at 1% false-alarm rate. Also: fix P.81 station coordinates (was Ban Pong/Ratchaburi, 493 km out of basin; now 18.6936 N 99.0819 E per RID station page), pin scikit-learn==1.9.0 and numpy<2, gitignore model artifacts (~100 MB, train on the server via scripts/train_flood_model.py).	2026-08-10 12:49:47 +07:00