Commit Graph
2 Commits
Author SHA1 Message Date
grabowski 0005f7dce1 feat: codified backtests, honest docs, belt-and-braces serving, perf fixes
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 13s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 23s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
Documentation / Validate Documentation (push) Failing after 7s
Documentation / Generate API Documentation (push) Successful in 8s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 3s
Retrained on the gap-filled DB (592k -> 976k rows) and re-examined the
flood backtests, now reproducible via scripts/backtest_render.py (renders
the three docs/img charts and gates on a >=12h 2024 first-alert lead —
currently failing by design and documented as such).

Findings, all documented in FLOOD_FORECASTING.md: the true 2024 crossing
was 24 Sep 17:00 (8h earlier than recorded; confirmed against the
independent HII sensor), the historical 24h-warning claim was partly a
missing-data artifact, and retrained warn classifiers collapse on the
filled grid (P.1 24h PR-AUC 0.900 -> 0.288) while regression MAE improves
(11.3 -> 10.5 cm). Serving therefore becomes max(classifier,
sigmoid(regression)) so alerting is never worse than the regression path;
metrics table, head-gating tiers, honest-limits and runbook expectations
all updated to the current model (hgb-v1+d2d0e65).

Perf, from Locust load testing (scripts/locustfile.py + load_test.py):
single-flight lock around /forecast inference (concurrent cache misses
previously each ran ~18s inference and starved the shared thread pool;
200-user run after: 105 rps, 0.01% errors), and /measurements/latest +
/health moved off the event loop (synchronous DB/network calls in async
handlers were stalling every request under load).
2026-08-12 10:46:00 +07:00
grabowski 27fa292e09 docs: September 2025 flood render - deployed config, 24 h warning, peak within 7 cm
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 25s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 12s
Documentation / Validate Documentation (push) Failing after 8s
Documentation / Generate API Documentation (push) Successful in 11s
Documentation / Build Sphinx Documentation (push) Successful in 16s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 0s
Documentation / Documentation Summary (push) Successful in 2s
Hour-by-hour chart of the 25-28 Sep 2025 event as forecast by the exact
deployed configuration (trained through 2024, event unseen): first alert
26 Sep 18:00, flooding began 27 Sep 18:00 (24 h lead), predicted peak
4.00 m vs actual 3.93 m. The near-miss 3.51 m crest on 26 Sep correctly
never alerted.
2026-08-10 18:35:55 +07:00