grabowski
6d4beae65a
CI / Format & lint (push) Successful in 24s
CI / Test suite (push) Successful in 48s
Docs / Validate documentation (push) Successful in 24s
Security / Dependency vulnerabilities (push) Successful in 2m20s
Security / Static analysis (push) Successful in 21s
Security / License report (push) Successful in 1m40s
build_features cut the rain series to the gauge grid before the forward sum, and the grid ends at the newest reading, so the row every live forecast is made from had rain_fc24 = NaN. Training and all evaluations had the real Open-Meteo forecast there. The forward sum now runs over the grid extended by 24 h; earlier rows are unchanged. Harness (new opt-in variant rise_rain_fc24_missing = rise_rain with rain_fc24 blanked on test rows, five folds, --from-cache): P.1 and P.103 nearly unaffected, P.21 false alarms 10 -> 5 and MAE 0.232 -> 0.215 with the fix. Write-up in docs/FLOOD_FORECASTING.md section 5; results in models/eval_2026-10-18_fc24.json. Also: HII mirror rows in one hour now resolve to the latest reading via a stable sort on the raw time (was an unstable sort on the floored hour, i.e. row-order dependent). Needed for an exact Rust port.
65 lines
2.1 KiB
Python
65 lines
2.1 KiB
Python
"""rain_fc24 at the newest grid row (the one serving predicts from)."""
|
|
|
|
import numpy as np
|
|
import pandas as pd
|
|
|
|
from src.ml import features
|
|
from src.ml.data import _prepare_hii_levels
|
|
|
|
|
|
def _grid(hours=48):
|
|
idx = pd.date_range("2026-09-01", periods=hours, freq="h")
|
|
df = pd.DataFrame(
|
|
{
|
|
"timestamp": idx,
|
|
"station_code": "P.1",
|
|
"water_level": np.linspace(1.0, 2.0, hours),
|
|
"discharge": np.nan,
|
|
}
|
|
)
|
|
return features.make_hourly_grid(df)
|
|
|
|
|
|
def test_forward_rain_reaches_past_the_last_reading():
|
|
grid = _grid()
|
|
idx = grid.observed.index
|
|
rain = pd.Series(
|
|
np.arange(1, len(idx) + 49, dtype=float),
|
|
index=pd.date_range(idx[0], periods=len(idx) + 48, freq="h"),
|
|
)
|
|
X = features.build_features(grid, "P.1", rain=rain)
|
|
last = idx[-1]
|
|
expected = rain.loc[last + pd.Timedelta(hours=1) : last + pd.Timedelta(hours=24)]
|
|
assert X.loc[last, "rain_fc24"] == expected.sum()
|
|
# rows whose window lies inside the grid are what they always were
|
|
old = rain.reindex(idx).shift(-1).iloc[::-1].rolling(24, min_periods=1).sum()
|
|
inner = idx[:-24]
|
|
np.testing.assert_allclose(X.loc[inner, "rain_fc24"], old.iloc[::-1].loc[inner])
|
|
|
|
|
|
def test_forward_rain_nan_without_forecast_hours():
|
|
grid = _grid()
|
|
idx = grid.observed.index
|
|
rain = pd.Series(1.0, index=idx) # nothing after the last reading
|
|
X = features.build_features(grid, "P.1", rain=rain)
|
|
assert np.isnan(X.loc[idx[-1], "rain_fc24"])
|
|
assert X.loc[idx[-2], "rain_fc24"] == 1.0
|
|
|
|
|
|
def test_hii_rows_latest_reading_in_an_hour_wins():
|
|
raw = pd.DataFrame(
|
|
{
|
|
"timestamp": pd.to_datetime(
|
|
["2026-10-01 10:40", "2026-10-01 10:10", "2026-10-01 11:00"]
|
|
),
|
|
"station_code": "P.1",
|
|
"wl_msl": [301.5, 301.2, 301.7],
|
|
"discharge": [10.0, 9.0, 11.0],
|
|
}
|
|
)
|
|
out = _prepare_hii_levels(raw)
|
|
assert list(out["wl_msl"]) == [301.5, 301.7]
|
|
assert list(out["timestamp"]) == list(
|
|
pd.to_datetime(["2026-10-01 10:00", "2026-10-01 11:00"])
|
|
)
|