fix(ml): serve the forecast rain the model was trained with
CI / Format & lint (push) Successful in 24s
CI / Test suite (push) Successful in 48s
Docs / Validate documentation (push) Successful in 24s
Security / Dependency vulnerabilities (push) Successful in 2m20s
Security / Static analysis (push) Successful in 21s
Security / License report (push) Successful in 1m40s
CI / Format & lint (push) Successful in 24s
CI / Test suite (push) Successful in 48s
Docs / Validate documentation (push) Successful in 24s
Security / Dependency vulnerabilities (push) Successful in 2m20s
Security / Static analysis (push) Successful in 21s
Security / License report (push) Successful in 1m40s
build_features cut the rain series to the gauge grid before the forward sum, and the grid ends at the newest reading, so the row every live forecast is made from had rain_fc24 = NaN. Training and all evaluations had the real Open-Meteo forecast there. The forward sum now runs over the grid extended by 24 h; earlier rows are unchanged. Harness (new opt-in variant rise_rain_fc24_missing = rise_rain with rain_fc24 blanked on test rows, five folds, --from-cache): P.1 and P.103 nearly unaffected, P.21 false alarms 10 -> 5 and MAE 0.232 -> 0.215 with the fix. Write-up in docs/FLOOD_FORECASTING.md section 5; results in models/eval_2026-10-18_fc24.json. Also: HII mirror rows in one hour now resolve to the latest reading via a stable sort on the raw time (was an unstable sort on the floored hour, i.e. row-order dependent). Needed for an exact Rust port.
This commit is contained in:
1 parent
64c5b02e06
commit
6d4beae65a
6 files changed
+876
-6
No files matched your search
@@ -584,6 +584,34 @@ span. `src/ml/hii_rain.py` builds the catchment mean and
|
||||
a 24 h-sum bias/MAE/correlation, so the two sources' relationship is on record
|
||||
by the time the 2027 fold (train ≤ 2027-04-30, test Jun–Nov 2027) can test it.
|
||||
|
||||
### 2026-10-18: forecast rain was missing at serving time — fixed
|
||||
|
||||
`build_features` cut the rain series to the gauge grid before taking the
|
||||
forward sum, and the grid ends at the newest reading. So the row serving
|
||||
predicts from always had `rain_fc24 = NaN`, although training and every
|
||||
evaluation above had the real value (the Open-Meteo forecast, already fetched
|
||||
with `forecast_days=2`). The model was answering as if no weather forecast
|
||||
existed. The forward sum now runs over the grid extended by 24 h; earlier rows
|
||||
are unchanged, so training is unaffected.
|
||||
|
||||
Cost of the bug, measured with the harness: the `rise_rain_fc24_missing`
|
||||
variant is `rise_rain` with `rain_fc24` blanked on the test rows (results in
|
||||
`models/eval_2026-10-18_fc24.json`, five monsoon folds, `--from-cache`):
|
||||
|
||||
| Station | Variant | Events | Warned early | Median lead | False alarms | MAE |
|
||||
|---|---|---|---|---|---|---|
|
||||
| P.1 | fixed (`rise_rain`) | 4 | 4 | 7.5 h | 0 | 0.087 |
|
||||
| P.1 | as served | 4 | 4 | 7.0 h | 0 | 0.087 |
|
||||
| P.103 | fixed | 7 | 7 | 9.0 h | 2 | 0.152 |
|
||||
| P.103 | as served | 7 | 7 | 9.0 h | 2 | 0.153 |
|
||||
| P.21 | fixed | 10 | 8 | 2.5 h | 5 | 0.215 |
|
||||
| P.21 | as served | 10 | 7 | 2.0 h | 10 | 0.232 |
|
||||
|
||||
P.1 and P.103 lost little; P.21 doubled its false alarms. The two very long
|
||||
"as served" leads (P.21 2025-08-26 +69 h, P.103 2025-10-03 +35 h) are alarm
|
||||
episodes that happened to precede an event, not real early warnings. Since the
|
||||
fix, the served features match what this section evaluates.
|
||||
|
||||
## 6. Deployment
|
||||
|
||||
### API
|
||||
|
||||
Reference in new issue
Block a user