feat: Open-Meteo catchment rainfall series + rain features + harness variant
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 25s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 17s
Documentation / Validate Documentation (push) Failing after 7s
Documentation / Generate API Documentation (push) Successful in 8s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 2s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 25s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 17s
Documentation / Validate Documentation (push) Failing after 7s
Documentation / Generate API Documentation (push) Successful in 8s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 2s
src/ml/rain.py fetches hourly precipitation for five upper-Ping catchment points (Chiang Dao, Mae Taeng, Mae Ngat, Mae Rim, city) from the Open-Meteo forecast-model archive (2021-03 onward, no API key, Bangkok-local timestamps, year-chunked local cache) plus the live forecast endpoint for serving (trailing days + next 48h). features.build_features/build_matrix accept the catchment-mean series and add rain_6h/24h/72h trailing sums and rain_fc24 — the forward 24h sum, a genuine forecast feature (archived forecasts at training time, a real weather forecast at serving; never contains river data). Columns exist only when a series is provided: HGB rejects all-NaN columns at fit, so no-rain training omits them and serving passes an empty series for alignment. evaluate.py gains the rise_rain variant (and --no-rain) so the harness can judge whether rain beats the deployed rise baseline.
This commit is contained in:
1 parent
21e9d2e114
commit
cbb3bf7369
3 files changed
+221
-6
No files matched your search
+30
-3
@@ -200,8 +200,24 @@ def _hours_since_observed(mask_col: pd.Series) -> pd.Series:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def build_features(grid: HourlyGrid, station: str) -> pd.DataFrame:
|
||||
"""Build the deterministic-order feature matrix for one target station."""
|
||||
RAIN_FEATURES = ("rain_6h", "rain_24h", "rain_72h", "rain_fc24")
|
||||
|
||||
|
||||
def build_features(
|
||||
grid: HourlyGrid, station: str, rain: Optional[pd.Series] = None
|
||||
) -> pd.DataFrame:
|
||||
"""Build the deterministic-order feature matrix for one target station.
|
||||
|
||||
``rain`` is the hourly catchment-average precipitation series (Open-Meteo,
|
||||
src/ml/rain.py). Rain columns are added only when a series is passed:
|
||||
HistGradientBoosting REJECTS all-NaN columns at fit time, so training
|
||||
without rain must omit the columns entirely (bundles record their
|
||||
feature_names, and serving subsets to them). At serving, pass an empty
|
||||
series rather than None so the columns exist (as NaN) for rain-trained
|
||||
bundles even when the live fetch fails. rain_fc24 is the forward 24 h
|
||||
sum: the archived forecast series at training time, a real weather
|
||||
forecast at serving time; it never contains river data.
|
||||
"""
|
||||
idx = grid.observed.index
|
||||
cols: Dict[str, pd.Series] = {}
|
||||
|
||||
@@ -254,6 +270,16 @@ def build_features(grid: HourlyGrid, station: str) -> pd.DataFrame:
|
||||
cols["doy_cos"] = np.cos(2 * np.pi * doy / 365.25)
|
||||
cols["is_monsoon"] = idx.to_series().dt.month.isin(MONSOON_MONTHS).astype(float)
|
||||
|
||||
if rain is not None:
|
||||
r = rain.reindex(idx)
|
||||
cols["rain_6h"] = r.rolling(6, min_periods=1).sum()
|
||||
cols["rain_24h"] = r.rolling(24, min_periods=1).sum()
|
||||
cols["rain_72h"] = r.rolling(72, min_periods=1).sum()
|
||||
# forward sum over (t, t+24]: shift(-1) starts the window at t+1
|
||||
cols["rain_fc24"] = (
|
||||
r.shift(-1).iloc[::-1].rolling(24, min_periods=1).sum().iloc[::-1]
|
||||
)
|
||||
|
||||
return pd.DataFrame(cols, index=idx)
|
||||
|
||||
|
||||
@@ -331,10 +357,11 @@ def build_matrix(
|
||||
station: str,
|
||||
horizons: Tuple[int, ...] = (6, 12, 24),
|
||||
stats_end: Optional[str] = None,
|
||||
rain: Optional[pd.Series] = None,
|
||||
) -> Tuple[pd.DataFrame, pd.DataFrame, dict]:
|
||||
"""Build (X, Y, meta) training/inference matrices for one station."""
|
||||
grid = make_hourly_grid(df_long)
|
||||
X = build_features(grid, station)
|
||||
X = build_features(grid, station, rain=rain)
|
||||
Y = build_labels(grid, station, horizons, stats_end=stats_end)
|
||||
|
||||
keep = X["obs_age_h"].notna()
|
||||
|
||||
Reference in new issue
Block a user