feat: Mae Ngat dam features — built, evaluated, defaulted OFF
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 27s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 15s
Documentation / Validate Documentation (push) Failing after 8s
Documentation / Generate API Documentation (push) Successful in 9s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 2s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 27s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 15s
Documentation / Validate Documentation (push) Failing after 8s
Documentation / Generate API Documentation (push) Successful in 9s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 2s
src/ml/dam.py loads rid_reservoir_daily into a leakage-safe hourly frame (daily row visible from 07:00 its own date, ffill capped at 48 h) and is plumbed through features/train/predict/evaluate exactly like rain, gated to the six mainstem stations below the Mae Ngat confluence. The experiment concludes as a documented NEGATIVE result: on the 2024 record-flood backtest every dam-feature subset costs 1-3 h of first-alert lead (13h -> 10-12h) for <=3 cm of peak-error gain, because the daily RID report lags up to 31 h and describes yesterday's benign absorbing reservoir during fast onset. Features therefore default OFF (--dam opt-in on the training and backtest CLIs; rise_rain_dam/rise_dam harness variants, excluded from the default variant set). The ablation also isolated the HII gap-fill as lead-neutral: the acceptance gate holds at 13 h with fill enabled, and docs/img charts are regenerated with the shipping configuration. Full table in docs/FLOOD_FORECASTING.md §5. Review-swarm fixes: evaluate.py skips variants whose feature family is absent instead of crashing the run; --dam forwards --db-url and warns loudly when no dam history loads; an empty DB result can no longer wipe a good dam cache; run-level metrics version claims v4 only when a dam station is actually in the set.
This commit is contained in:
1 parent
6af6fbe02c
commit
28b62e5a36
12 files changed
+464
-25
No files matched your search
+29
-2
@@ -202,9 +202,18 @@ def _hours_since_observed(mask_col: pd.Series) -> pd.Series:
|
||||
|
||||
RAIN_FEATURES = ("rain_6h", "rain_24h", "rain_72h", "rain_fc24")
|
||||
|
||||
DAM_FEATURES = ("dam_storage_pct", "dam_storage_pct_d3", "dam_inflow", "dam_outflow")
|
||||
# Stations hydrologically downstream of the Mae Ngat confluence (Ping mainstem
|
||||
# at/below Mae Taeng) — the only ones where reservoir state is causal. West-
|
||||
# tributary and upper-mainstem stations never receive dam columns.
|
||||
DAM_STATIONS = frozenset({"P.1", "P.103", "P.67", "P.21", "P.5", "P.81"})
|
||||
|
||||
|
||||
def build_features(
|
||||
grid: HourlyGrid, station: str, rain: Optional[pd.Series] = None
|
||||
grid: HourlyGrid,
|
||||
station: str,
|
||||
rain: Optional[pd.Series] = None,
|
||||
dam: Optional[pd.DataFrame] = None,
|
||||
) -> pd.DataFrame:
|
||||
"""Build the deterministic-order feature matrix for one target station.
|
||||
|
||||
@@ -217,6 +226,11 @@ def build_features(
|
||||
bundles even when the live fetch fails. rain_fc24 is the forward 24 h
|
||||
sum: the archived forecast series at training time, a real weather
|
||||
forecast at serving time; it never contains river data.
|
||||
|
||||
``dam`` is the hourly Mae Ngat reservoir frame (src/ml/dam.py; columns
|
||||
storage_pct/inflow_mcm/outflow_mcm, already leakage-shifted to 07:00
|
||||
report time). Same contract as rain: None omits the columns, an empty
|
||||
frame yields NaN columns; only DAM_STATIONS receive them.
|
||||
"""
|
||||
idx = grid.observed.index
|
||||
cols: Dict[str, pd.Series] = {}
|
||||
@@ -280,6 +294,18 @@ def build_features(
|
||||
r.shift(-1).iloc[::-1].rolling(24, min_periods=1).sum().iloc[::-1]
|
||||
)
|
||||
|
||||
if dam is not None and station in DAM_STATIONS:
|
||||
d = dam.reindex(idx)
|
||||
|
||||
def _dam_col(name: str) -> pd.Series:
|
||||
return d[name] if name in d.columns else pd.Series(np.nan, index=idx)
|
||||
|
||||
storage = _dam_col("storage_pct")
|
||||
cols["dam_storage_pct"] = storage
|
||||
cols["dam_storage_pct_d3"] = storage - storage.shift(72)
|
||||
cols["dam_inflow"] = _dam_col("inflow_mcm")
|
||||
cols["dam_outflow"] = _dam_col("outflow_mcm")
|
||||
|
||||
return pd.DataFrame(cols, index=idx)
|
||||
|
||||
|
||||
@@ -358,10 +384,11 @@ def build_matrix(
|
||||
horizons: Tuple[int, ...] = (6, 12, 24),
|
||||
stats_end: Optional[str] = None,
|
||||
rain: Optional[pd.Series] = None,
|
||||
dam: Optional[pd.DataFrame] = None,
|
||||
) -> Tuple[pd.DataFrame, pd.DataFrame, dict]:
|
||||
"""Build (X, Y, meta) training/inference matrices for one station."""
|
||||
grid = make_hourly_grid(df_long)
|
||||
X = build_features(grid, station, rain=rain)
|
||||
X = build_features(grid, station, rain=rain, dam=dam)
|
||||
Y = build_labels(grid, station, horizons, stats_end=stats_end)
|
||||
|
||||
keep = X["obs_age_h"].notna()
|
||||
|
||||
Reference in new issue
Block a user