eval: quantile heads and fc48 on top of hgb-v3 (rejected/deferred); HII gauge-rain aggregate
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 41s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 17s
Documentation / Validate Documentation (push) Failing after 16s
Documentation / Generate API Documentation (push) Successful in 11s
Documentation / Build Sphinx Documentation (push) Successful in 18s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 3s

Rolling-origin harness gains rise_rain_quantile, rise_rain_quantile_uw,
rise_rain_qsigma (L2 point + quantile sigma) and rise_rain_fc48, all
opt-in, plus --from-cache for reproducible offline reruns. Results in
models/eval_2026-09-12*.json, write-up in docs/FLOOD_FORECASTING.md:

- quantile point prediction: better MAE, worse first-alert lead at 5 of
  11 events (P.103 2022-08-14 +6h -> +1h) -> rejected
- quantile sigma only: Brier within noise (0.0031 -> 0.0029) -> not worth 3x heads
- rain_fc48: neutral everywhere except 2024-10-03 P.1 (+21h -> +72h), n=1
  -> deferred to after the 2026 season

src/ml/hii_rain.py: catchment-mean hourly rain from the ~130 HII gauges in
the upper-Ping box and a 24h-sum comparison against Open-Meteo. Not a
training feature (table exists only since 2026-08-11, no archive); exposed
at GET /api/hii/rainfall/catchment so the two sources' agreement is on
record by the time a fold can test it.

data._read_cache now skips non-station files in models/cache/ (the shared
dir also holds rain_openmeteo / dam_* caches, which crashed the reader).
scripts/summarize_eval.py prints per-variant lead/peak-error tables.
This commit is contained in:
2026-09-11 21:55:37 +02:00
parent 764764e07e
commit d621aa9ce7
10 changed files with 1607 additions and 7 deletions
+119
View File
@@ -0,0 +1,119 @@
"""Catchment-mean hourly rain from the HII/ThaiWater gauge network.
Independent of Open-Meteo (src/ml/rain.py): those are model-analysis values,
these are what the gauges measured. The `hii_rainfall` table has been filled
by the hourly collector since 2026-08-11 and there is NO archive behind it
(the api-v3 rain_24h_graph endpoint ignores its date range, see
docs/DATA_SOURCES.md 2.1), so this series cannot yet be a training feature:
every training row before 2026-08 would be NaN and HistGradientBoosting
would learn nothing from the column. It becomes a candidate once a full
monsoon season of gauge rows exists in the rolling-origin harness's test
span -- the 2027 fold (train through 2027-04-30, test Jun-Nov 2027) is the
first that could show anything.
Until then it serves two purposes:
* a live cross-check of the Open-Meteo catchment mean (/api/hii/rainfall
already exposes the raw gauges; this gives the comparable aggregate);
* accumulating the comparison so the eventual feature evaluation has a
documented bias/variance relationship between the two sources.
"""
import logging
from typing import Optional, Sequence, Tuple
import pandas as pd
from .data import resolve_db_url
logger = logging.getLogger(__name__)
# Same footprint as rain.CATCHMENT_POINTS: the upper Ping above P.1. Gauges
# inside this box are averaged; there are ~130 with recent data (DWR, FOP,
# HII, RID, TMD), far denser than the five Open-Meteo points.
CATCHMENT_BOX: Tuple[float, float, float, float] = (18.75, 19.60, 98.60, 99.30)
# A gauge that reports the same rain_24h for many hours is stuck; drop hours
# where fewer than this many gauges reported at all.
MIN_GAUGES_PER_HOUR = 5
def load_gauge_mean(
db_url: Optional[str] = None,
start: Optional[pd.Timestamp] = None,
end: Optional[pd.Timestamp] = None,
box: Sequence[float] = CATCHMENT_BOX,
engine=None,
) -> Optional[pd.Series]:
"""Hourly catchment-mean rain_1h (mm) across HII gauges in `box`.
Pass `engine` (the API's HII store engine) to reuse a pool; otherwise a
connection is resolved from db_url / config. Returns None if the DB is
unavailable or the table is empty. Hours with fewer than
MIN_GAUGES_PER_HOUR reporting gauges are NaN.
"""
if engine is None:
resolved = resolve_db_url(db_url)
if not resolved:
return None
lat_lo, lat_hi, lon_lo, lon_hi = box
try:
from sqlalchemy import create_engine, text
query = (
"SELECT m.timestamp, COUNT(m.rain_1h) AS n, AVG(m.rain_1h) AS rain_1h "
"FROM hii_rainfall m JOIN hii_rain_stations s ON s.id = m.station_id "
"WHERE s.latitude BETWEEN :lat_lo AND :lat_hi "
"AND s.longitude BETWEEN :lon_lo AND :lon_hi "
"AND m.rain_1h IS NOT NULL"
)
params = {"lat_lo": lat_lo, "lat_hi": lat_hi, "lon_lo": lon_lo, "lon_hi": lon_hi}
if start is not None:
query += " AND m.timestamp >= :start"
params["start"] = pd.Timestamp(start).to_pydatetime()
if end is not None:
query += " AND m.timestamp <= :end"
params["end"] = pd.Timestamp(end).to_pydatetime()
query += " GROUP BY m.timestamp ORDER BY m.timestamp"
if engine is None:
engine = create_engine(resolved, pool_pre_ping=True)
with engine.connect() as conn:
frame = pd.read_sql(text(query), conn, params=params)
except Exception as error:
logger.warning(f"HII gauge rain load failed: {error}")
return None
if frame.empty:
return None
frame["timestamp"] = pd.to_datetime(frame["timestamp"]).dt.floor("h")
frame = frame.groupby("timestamp").agg(n=("n", "sum"), rain_1h=("rain_1h", "mean"))
series = pd.to_numeric(frame["rain_1h"], errors="coerce")
series[frame["n"] < MIN_GAUGES_PER_HOUR] = float("nan")
series.name = "hii_gauge_mean"
return series
def compare_with_openmeteo(
gauge: pd.Series, openmeteo: pd.Series, window_h: int = 24
) -> dict:
"""Bias/correlation of Open-Meteo against the gauges over the overlap.
Both are summed over trailing `window_h` so single-hour timing offsets
(gauges report at :00, the model's hour is an interval) do not dominate.
"""
joined = pd.concat(
{"gauge": gauge, "openmeteo": openmeteo}, axis=1
).dropna()
if joined.empty:
return {"overlap_hours": 0}
g = joined["gauge"].rolling(window_h, min_periods=window_h).sum()
o = joined["openmeteo"].rolling(window_h, min_periods=window_h).sum()
both = pd.concat({"g": g, "o": o}, axis=1).dropna()
if both.empty:
return {"overlap_hours": int(len(joined))}
return {
"overlap_hours": int(len(joined)),
"window_h": window_h,
"gauge_mean_mm": float(both["g"].mean()),
"openmeteo_mean_mm": float(both["o"].mean()),
"bias_mm": float((both["o"] - both["g"]).mean()),
"mae_mm": float((both["o"] - both["g"]).abs().mean()),
"corr": float(both["g"].corr(both["o"])),
}