eval: quantile heads and fc48 on top of hgb-v3 (rejected/deferred); HII gauge-rain aggregate
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 41s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 17s
Documentation / Validate Documentation (push) Failing after 16s
Documentation / Generate API Documentation (push) Successful in 11s
Documentation / Build Sphinx Documentation (push) Successful in 18s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 3s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 41s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 17s
Documentation / Validate Documentation (push) Failing after 16s
Documentation / Generate API Documentation (push) Successful in 11s
Documentation / Build Sphinx Documentation (push) Successful in 18s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 3s
Rolling-origin harness gains rise_rain_quantile, rise_rain_quantile_uw, rise_rain_qsigma (L2 point + quantile sigma) and rise_rain_fc48, all opt-in, plus --from-cache for reproducible offline reruns. Results in models/eval_2026-09-12*.json, write-up in docs/FLOOD_FORECASTING.md: - quantile point prediction: better MAE, worse first-alert lead at 5 of 11 events (P.103 2022-08-14 +6h -> +1h) -> rejected - quantile sigma only: Brier within noise (0.0031 -> 0.0029) -> not worth 3x heads - rain_fc48: neutral everywhere except 2024-10-03 P.1 (+21h -> +72h), n=1 -> deferred to after the 2026 season src/ml/hii_rain.py: catchment-mean hourly rain from the ~130 HII gauges in the upper-Ping box and a 24h-sum comparison against Open-Meteo. Not a training feature (table exists only since 2026-08-11, no archive); exposed at GET /api/hii/rainfall/catchment so the two sources' agreement is on record by the time a fold can test it. data._read_cache now skips non-station files in models/cache/ (the shared dir also holds rain_openmeteo / dam_* caches, which crashed the reader). scripts/summarize_eval.py prints per-variant lead/peak-error tables.
This commit is contained in:
@@ -353,6 +353,29 @@ class TestHiiApiEndpoints:
|
||||
self._get(web_api, "get_hii_rainfall_latest", hours=48)
|
||||
assert calls["n"] == 2
|
||||
|
||||
def test_rainfall_catchment(self, web_api):
|
||||
"""One gauge in the box (CHM005, 19.12N 98.94E) is below the
|
||||
MIN_GAUGES_PER_HOUR floor, so the catchment mean is NaN -> null, the
|
||||
openmeteo_rain table does not exist in this store, and the comparison
|
||||
reports no overlap. Shape is what matters: the endpoint must not 500
|
||||
on a fresh database."""
|
||||
payload, response = self._get(web_api, "get_hii_rainfall_catchment", days=7)
|
||||
assert "x-data-stale" not in response.headers
|
||||
assert list(payload) == ["box", "gauge", "openmeteo", "comparison_24h_sums"]
|
||||
assert payload["openmeteo"] == []
|
||||
assert payload["comparison_24h_sums"] == {"overlap_hours": 0}
|
||||
assert len(payload["gauge"]) == 1
|
||||
assert payload["gauge"][0]["rain_mm"] is None # < MIN_GAUGES_PER_HOUR
|
||||
|
||||
def test_rainfall_catchment_disabled(self, monkeypatch):
|
||||
from src import web_api
|
||||
|
||||
monkeypatch.setitem(web_api.app_state, "hii_collector", None)
|
||||
web_api.HII_CACHE.clear()
|
||||
web_api._REFRESH_IN_FLIGHT.clear()
|
||||
payload, _ = self._get(web_api, "get_hii_rainfall_catchment", days=7)
|
||||
assert payload["gauge"] == [] and payload["openmeteo"] == []
|
||||
|
||||
def test_stale_served_on_recompute_failure(self, web_api, monkeypatch):
|
||||
# Prime the cache, expire it, break the DB: the stale copy is served
|
||||
# and flagged via the X-Data-Stale header.
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
"""HII gauge-rain aggregate: pure-function tests (no DB)."""
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from src.ml import hii_rain
|
||||
|
||||
|
||||
def _hourly(start, n):
|
||||
return pd.date_range(start, periods=n, freq="h")
|
||||
|
||||
|
||||
def test_compare_identical_series_has_zero_bias():
|
||||
idx = _hourly("2026-08-12", 200)
|
||||
rng = np.random.default_rng(1)
|
||||
rain = pd.Series(rng.exponential(0.5, len(idx)), index=idx)
|
||||
out = hii_rain.compare_with_openmeteo(rain, rain.copy(), window_h=24)
|
||||
assert out["overlap_hours"] == 200
|
||||
assert out["bias_mm"] == 0.0
|
||||
assert out["mae_mm"] == 0.0
|
||||
assert out["corr"] > 0.999
|
||||
|
||||
|
||||
def test_compare_reports_constant_bias():
|
||||
idx = _hourly("2026-08-12", 100)
|
||||
gauge = pd.Series(1.0, index=idx)
|
||||
model = pd.Series(1.5, index=idx) # model wetter by 0.5 mm/h
|
||||
out = hii_rain.compare_with_openmeteo(gauge, model, window_h=24)
|
||||
assert abs(out["bias_mm"] - 12.0) < 1e-9 # 0.5 mm/h x 24 h
|
||||
|
||||
|
||||
def test_compare_uses_overlap_only():
|
||||
gauge = pd.Series(1.0, index=_hourly("2026-08-12", 100))
|
||||
model = pd.Series(1.0, index=_hourly("2026-08-14", 100)) # 52 h overlap
|
||||
out = hii_rain.compare_with_openmeteo(gauge, model, window_h=24)
|
||||
assert out["overlap_hours"] == 52
|
||||
|
||||
|
||||
def test_compare_no_overlap():
|
||||
gauge = pd.Series(1.0, index=_hourly("2026-01-01", 10))
|
||||
model = pd.Series(1.0, index=_hourly("2026-06-01", 10))
|
||||
assert hii_rain.compare_with_openmeteo(gauge, model) == {"overlap_hours": 0}
|
||||
|
||||
|
||||
def test_load_gauge_mean_without_db_returns_none(monkeypatch):
|
||||
monkeypatch.setattr(hii_rain, "resolve_db_url", lambda *a, **k: None)
|
||||
assert hii_rain.load_gauge_mean() is None
|
||||
Reference in New Issue
Block a user