feat: Mae Ngat dam features — built, evaluated, defaulted OFF
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 27s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 15s
Documentation / Validate Documentation (push) Failing after 8s
Documentation / Generate API Documentation (push) Successful in 9s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 2s

src/ml/dam.py loads rid_reservoir_daily into a leakage-safe hourly frame
(daily row visible from 07:00 its own date, ffill capped at 48 h) and is
plumbed through features/train/predict/evaluate exactly like rain, gated
to the six mainstem stations below the Mae Ngat confluence.

The experiment concludes as a documented NEGATIVE result: on the 2024
record-flood backtest every dam-feature subset costs 1-3 h of first-alert
lead (13h -> 10-12h) for <=3 cm of peak-error gain, because the daily RID
report lags up to 31 h and describes yesterday's benign absorbing
reservoir during fast onset. Features therefore default OFF (--dam
opt-in on the training and backtest CLIs; rise_rain_dam/rise_dam harness
variants, excluded from the default variant set). The ablation also
isolated the HII gap-fill as lead-neutral: the acceptance gate holds at
13 h with fill enabled, and docs/img charts are regenerated with the
shipping configuration. Full table in docs/FLOOD_FORECASTING.md §5.

Review-swarm fixes: evaluate.py skips variants whose feature family is
absent instead of crashing the run; --dam forwards --db-url and warns
loudly when no dam history loads; an empty DB result can no longer wipe
a good dam cache; run-level metrics version claims v4 only when a dam
station is actually in the set.
This commit is contained in:
grabowski committed 2026-08-13 20:42:21 +07:00
1 parent 6af6fbe02c
commit 28b62e5a36
12 files changed
+464 -25

No files matched your search

+29 -2
View File
@@ -202,9 +202,18 @@ def _hours_since_observed(mask_col: pd.Series) -> pd.Series:
RAIN_FEATURES = ("rain_6h", "rain_24h", "rain_72h", "rain_fc24")
DAM_FEATURES = ("dam_storage_pct", "dam_storage_pct_d3", "dam_inflow", "dam_outflow")
# Stations hydrologically downstream of the Mae Ngat confluence (Ping mainstem
# at/below Mae Taeng) — the only ones where reservoir state is causal. West-
# tributary and upper-mainstem stations never receive dam columns.
DAM_STATIONS = frozenset({"P.1", "P.103", "P.67", "P.21", "P.5", "P.81"})
def build_features(
grid: HourlyGrid, station: str, rain: Optional[pd.Series] = None
grid: HourlyGrid,
station: str,
rain: Optional[pd.Series] = None,
dam: Optional[pd.DataFrame] = None,
) -> pd.DataFrame:
"""Build the deterministic-order feature matrix for one target station.
@@ -217,6 +226,11 @@ def build_features(
bundles even when the live fetch fails. rain_fc24 is the forward 24 h
sum: the archived forecast series at training time, a real weather
forecast at serving time; it never contains river data.
``dam`` is the hourly Mae Ngat reservoir frame (src/ml/dam.py; columns
storage_pct/inflow_mcm/outflow_mcm, already leakage-shifted to 07:00
report time). Same contract as rain: None omits the columns, an empty
frame yields NaN columns; only DAM_STATIONS receive them.
"""
idx = grid.observed.index
cols: Dict[str, pd.Series] = {}
@@ -280,6 +294,18 @@ def build_features(
r.shift(-1).iloc[::-1].rolling(24, min_periods=1).sum().iloc[::-1]
)
if dam is not None and station in DAM_STATIONS:
d = dam.reindex(idx)
def _dam_col(name: str) -> pd.Series:
return d[name] if name in d.columns else pd.Series(np.nan, index=idx)
storage = _dam_col("storage_pct")
cols["dam_storage_pct"] = storage
cols["dam_storage_pct_d3"] = storage - storage.shift(72)
cols["dam_inflow"] = _dam_col("inflow_mcm")
cols["dam_outflow"] = _dam_col("outflow_mcm")
return pd.DataFrame(cols, index=idx)
@@ -358,10 +384,11 @@ def build_matrix(
horizons: Tuple[int, ...] = (6, 12, 24),
stats_end: Optional[str] = None,
rain: Optional[pd.Series] = None,
dam: Optional[pd.DataFrame] = None,
) -> Tuple[pd.DataFrame, pd.DataFrame, dict]:
"""Build (X, Y, meta) training/inference matrices for one station."""
grid = make_hourly_grid(df_long)
X = build_features(grid, station, rain=rain)
X = build_features(grid, station, rain=rain, dam=dam)
Y = build_labels(grid, station, horizons, stats_end=stats_end)
keep = X["obs_age_h"].notna()