5 Commits

Include renames

Author	SHA1	Message	Date
grabowski	28b62e5a36	feat: Mae Ngat dam features — built, evaluated, defaulted OFF ... src/ml/dam.py loads rid_reservoir_daily into a leakage-safe hourly frame (daily row visible from 07:00 its own date, ffill capped at 48 h) and is plumbed through features/train/predict/evaluate exactly like rain, gated to the six mainstem stations below the Mae Ngat confluence. The experiment concludes as a documented NEGATIVE result: on the 2024 record-flood backtest every dam-feature subset costs 1-3 h of first-alert lead (13h -> 10-12h) for <=3 cm of peak-error gain, because the daily RID report lags up to 31 h and describes yesterday's benign absorbing reservoir during fast onset. Features therefore default OFF (--dam opt-in on the training and backtest CLIs; rise_rain_dam/rise_dam harness variants, excluded from the default variant set). The ablation also isolated the HII gap-fill as lead-neutral: the acceptance gate holds at 13 h with fill enabled, and docs/img charts are regenerated with the shipping configuration. Full table in docs/FLOOD_FORECASTING.md §5. Review-swarm fixes: evaluate.py skips variants whose feature family is absent instead of crashing the run; --dam forwards --db-url and warns loudly when no dam history loads; an empty DB result can no longer wipe a good dam cache; run-level metrics version claims v4 only when a dam station is actually in the set.	2026-08-13 20:42:21 +07:00
grabowski	df0ae8cda3	feat: hgb-v3 — Open-Meteo rain features clear the 12h warning gate ... The rolling-origin harness (models/eval_rain.json) showed catchment rain halving flood-year Brier scores, cutting flood-regime MAE 20-40%, and extending the hard 2024 leads (+6h -> +11h at P.1, +10h -> +19h at P.103). Ported: train_all loads the catchment-mean series (use_rain / --no-rain to opt out; without it bundles train as v2), predict fetches live rain hourly and passes an empty series on failure so rain-trained bundles serve with NaN features instead of tripping the feature guard, and the leader worker persists hourly per-point + catchment-mean rows to a new openmeteo_rain table. Regenerated backtest: the 2024 record flood now gets a 13-HOUR WARNING (alert 04:00 vs 17:00 crossing, river at 2.9m at alert time) — the >=12h acceptance gate PASSES for the first time. Journey on that crossing: v1 -18h, v2 +6h, v3 +13h. The marginal 2025 double-crest trades its artifact +46h latch for a calibrated +2h with zero false alarms. P.1 MAE 4.9/7.2/8.7 cm at 6/12/24h. Docs updated throughout.	2026-08-12 17:05:01 +07:00
grabowski	21e9d2e114	feat: hgb-v2 — regression heads predict rise, recovering flood warning lead ... Rolling-origin evaluation (5 monsoon folds x 4 variants, P.1 + P.103; results in models/eval_variants.json) showed the absolute-level target alerting AT the crossing on essentially every event, while the rise target (future max - current level, level added back at serving) gives +6h on the hard 2024 crossings, +45h in 2025, fewer false alarms than weighted/quantile variants, and ~11% better MAE. Weighted and quantile variants rejected: more false alarms, no Brier-score calibration gain. Ported to production: train.py fits rise in both eval and refit passes (sigma/metrics computed in absolute space), bundles stamped hgb-v2 with regression_target='rise', predict.py adds the level back for v2 and stays compatible with v1 bundles, backtest_render.py mirrors the same math. Regenerated backtest charts: 2024 first alert 11:00 24 Sep (6h BEFORE the 17:00 crossing, was 18h after), 2025 alert 45h ahead, and the record-peak underprediction is gone (rise models can exceed the training max). The >=12h acceptance gate still fails honestly at +6h — closing that needs rainfall inputs. New P.1 MAE 5.0/7.2/9.4 cm at 6/12/24h; docs updated throughout.	2026-08-12 15:43:29 +07:00
grabowski	0005f7dce1	feat: codified backtests, honest docs, belt-and-braces serving, perf fixes ... Retrained on the gap-filled DB (592k -> 976k rows) and re-examined the flood backtests, now reproducible via scripts/backtest_render.py (renders the three docs/img charts and gates on a >=12h 2024 first-alert lead — currently failing by design and documented as such). Findings, all documented in FLOOD_FORECASTING.md: the true 2024 crossing was 24 Sep 17:00 (8h earlier than recorded; confirmed against the independent HII sensor), the historical 24h-warning claim was partly a missing-data artifact, and retrained warn classifiers collapse on the filled grid (P.1 24h PR-AUC 0.900 -> 0.288) while regression MAE improves (11.3 -> 10.5 cm). Serving therefore becomes max(classifier, sigmoid(regression)) so alerting is never worse than the regression path; metrics table, head-gating tiers, honest-limits and runbook expectations all updated to the current model (hgb-v1+d2d0e65). Perf, from Locust load testing (scripts/locustfile.py + load_test.py): single-flight lock around /forecast inference (concurrent cache misses previously each ran ~18s inference and starved the shared thread pool; 200-user run after: 105 rps, 0.01% errors), and /measurements/latest + /health moved off the event loop (synchronous DB/network calls in async handlers were stalling every request under load).	2026-08-12 10:46:00 +07:00
grabowski	27fa292e09	docs: September 2025 flood render - deployed config, 24 h warning, peak within 7 cm ... Hour-by-hour chart of the 25-28 Sep 2025 event as forecast by the exact deployed configuration (trained through 2024, event unseen): first alert 26 Sep 18:00, flooding began 27 Sep 18:00 (24 h lead), predicted peak 4.00 m vs actual 3.93 m. The near-miss 3.51 m crest on 26 Sep correctly never alerted.	2026-08-10 18:35:55 +07:00