feat: ML flood-event forecasting from 8 years of gauge history
Security & Dependency Updates / Dependency Security Scan (push) Successful in 1m8s
Security & Dependency Updates / License Compliance (push) Successful in 25s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 20s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 17s
Security & Dependency Updates / Security Summary (push) Successful in 9s
Security & Dependency Updates / Dependency Security Scan (push) Successful in 1m8s
Security & Dependency Updates / License Compliance (push) Successful in 25s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 20s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 17s
Security & Dependency Updates / Security Summary (push) Successful in 9s
Add src/ml/ package predicting, per station and per 6/12/24 h horizon, the probability of exceeding warning (3.0 m) and danger (4.5 m) levels plus expected peak level, trained on the 592k-row PostgreSQL history: - features.py: hourly grid with coverage gating and no future leakage; upstream stations enter at empirically measured travel-time lags (P.20 +17h ... P.103 +1h vs P.1); hour-of-day deliberately excluded (it encodes the scrape schedule, not hydrology) - train.py: HistGradientBoosting regression + warn/danger classifier heads per station x horizon, >=30-positives gate with calibrated sigmoid-on-regression fallback, strict temporal splits, per-event lead-time evaluation; guards against sklearn 1.9.0 crash on degenerate feature columns - predict.py: bundle loading with feature-name checks, heuristic fallback tier, get_latest_forecasts() for the API; raises when no models are trained so the endpoint 503s instead of serving persistence output as forecasts - data.py: Postgres-first loader (FLOOD_ML_DB_URL override), HTTP API fallback (flagged: that path backfills synthetic discharge), csv.gz cache - /forecast endpoint (15-min TTL cache) + dashboard flood-risk panel (hidden until models exist) - docs/FLOOD_FORECASTING.md: full system doc with measured deployment numbers (~335 MB RSS, CPU negligible, ~6 min full retrain) and retraining policy Validation: out-of-sample backtest of the record 2024 flood season (train <= Aug 2024) alerted 24-48 h ahead of the Oct 5 peak; 2025-26 test split: P.1 6h PR-AUC 0.974, recall 98.3% at 1% false-alarm rate. Also: fix P.81 station coordinates (was Ban Pong/Ratchaburi, 493 km out of basin; now 18.6936 N 99.0819 E per RID station page), pin scikit-learn==1.9.0 and numpy<2, gitignore model artifacts (~100 MB, train on the server via scripts/train_flood_model.py).
This commit is contained in:
@@ -41,6 +41,10 @@ HISTORY_CACHE: Dict[str, tuple] = {}
|
||||
HISTORY_CACHE_LOCK = Lock()
|
||||
HISTORY_TTL = 300 # 5 minutes
|
||||
|
||||
FORECAST_CACHE: Dict[str, tuple] = {}
|
||||
FORECAST_CACHE_LOCK = Lock()
|
||||
FORECAST_TTL = 900 # 15 minutes
|
||||
|
||||
# Dashboard HTML is loaded once at import from src/static/dashboard.html.
|
||||
_DASHBOARD_HTML_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), "static", "dashboard.html")
|
||||
try:
|
||||
@@ -495,6 +499,33 @@ async def get_postgres_history(
|
||||
raise HTTPException(status_code=502, detail="Measurement history unavailable")
|
||||
|
||||
|
||||
@app.get("/forecast")
|
||||
async def get_flood_forecasts():
|
||||
"""Flood-risk forecasts per station for the 6/12/24 h horizons."""
|
||||
increment_counter("api_requests", labels={"endpoint": "forecast"})
|
||||
now = time.monotonic()
|
||||
with FORECAST_CACHE_LOCK:
|
||||
cached = FORECAST_CACHE.get("all")
|
||||
if cached and now - cached[0] < FORECAST_TTL:
|
||||
return cached[1]
|
||||
try:
|
||||
from .ml.predict import get_latest_forecasts
|
||||
except ImportError as error:
|
||||
raise HTTPException(status_code=503, detail=f"Forecasting unavailable: {error}")
|
||||
try:
|
||||
data = await asyncio.to_thread(get_latest_forecasts)
|
||||
except FileNotFoundError:
|
||||
raise HTTPException(status_code=503, detail="No trained flood models found")
|
||||
except RuntimeError as error:
|
||||
raise HTTPException(status_code=503, detail=str(error))
|
||||
except Exception as error:
|
||||
logger.error(f"Error computing flood forecasts: {error}")
|
||||
raise HTTPException(status_code=502, detail="Flood forecast unavailable")
|
||||
with FORECAST_CACHE_LOCK:
|
||||
FORECAST_CACHE["all"] = (now, data)
|
||||
return data
|
||||
|
||||
|
||||
@app.get("/measurements/latest", response_model=List[MeasurementResponse])
|
||||
async def get_latest_measurements(limit: int = 100):
|
||||
"""Get latest measurements from all stations"""
|
||||
|
||||
Reference in New Issue
Block a user