Replace the network-wide (3.0, 4.5) m thresholds with per-station values
calibrated from the DB's discharge_percent (RID % of channel capacity):
warning = median level at 75-85% capacity, danger = median at 95-105%.
Fixes P.103 over-alerting (bank-full ~6.75 m, not 4.5) and P.67
under-alerting (overflow ~2.9 m). Requires a retrain to take effect in
the classifier heads.
P.1 uses the official Chiang Mai municipal inundation map instead:
warning 3.70 m (stage 1, city flooding begins), danger 4.20 m (stage 5),
with the full 7-stage table (3.70-4.60 m + discharge) in
features.P1_FLOOD_STAGES. Forecast rows for P.1 now include per-stage
exceedance probabilities computed from the regression head + calibration
sigma - available immediately without retraining.
Dashboard: "Chiang Mai city flood outlook" block above the forecast grid
(predicted peak + 7 stage-probability chips) and a toggleable
georeferenced overlay of the official flood-zone map
(static/flood-zones-p1.jpg, bounds tunable in FLOOD_ZONE_BOUNDS).
Add src/ml/ package predicting, per station and per 6/12/24 h horizon,
the probability of exceeding warning (3.0 m) and danger (4.5 m) levels
plus expected peak level, trained on the 592k-row PostgreSQL history:
- features.py: hourly grid with coverage gating and no future leakage;
upstream stations enter at empirically measured travel-time lags
(P.20 +17h ... P.103 +1h vs P.1); hour-of-day deliberately excluded
(it encodes the scrape schedule, not hydrology)
- train.py: HistGradientBoosting regression + warn/danger classifier
heads per station x horizon, >=30-positives gate with calibrated
sigmoid-on-regression fallback, strict temporal splits, per-event
lead-time evaluation; guards against sklearn 1.9.0 crash on
degenerate feature columns
- predict.py: bundle loading with feature-name checks, heuristic
fallback tier, get_latest_forecasts() for the API; raises when no
models are trained so the endpoint 503s instead of serving
persistence output as forecasts
- data.py: Postgres-first loader (FLOOD_ML_DB_URL override), HTTP API
fallback (flagged: that path backfills synthetic discharge), csv.gz
cache
- /forecast endpoint (15-min TTL cache) + dashboard flood-risk panel
(hidden until models exist)
- docs/FLOOD_FORECASTING.md: full system doc with measured deployment
numbers (~335 MB RSS, CPU negligible, ~6 min full retrain) and
retraining policy
Validation: out-of-sample backtest of the record 2024 flood season
(train <= Aug 2024) alerted 24-48 h ahead of the Oct 5 peak; 2025-26
test split: P.1 6h PR-AUC 0.974, recall 98.3% at 1% false-alarm rate.
Also: fix P.81 station coordinates (was Ban Pong/Ratchaburi, 493 km
out of basin; now 18.6936 N 99.0819 E per RID station page), pin
scikit-learn==1.9.0 and numpy<2, gitignore model artifacts (~100 MB,
train on the server via scripts/train_flood_model.py).
Cover the previously-untested, highest-risk parsing in
fetch_water_data_for_date by mocking the HTTP call:
- hour 1-23 map to the same day; hour 24 rolls to next-day midnight
- qvalues "***" / None yield discharge None (no crash)
- None water level is skipped
- out-of-range (0, 25) and empty hourlytime rows are skipped
- missing "rows" key returns an empty list
This gives the scraper a safety net before its module is split.
- tests/conftest.py: put repo root on sys.path so `import src...` resolves
under pytest regardless of invocation directory.
- test_matrix_formatting.py: lock in HTML formatted_body + plain-text fallback,
URL linkification, HTML escaping, and send_alert field rendering.
- test_station_persistence.py: cover default-load, save/reload round-trip
(incl. Thai text), runtime-file precedence, and atomic-write cleanup.
These are real assert-based tests (unlike the existing print-style scripts) so
CI can gate on them. 13 tests, all passing.
- Created test suite for zone-based water level alerts (9 test cases)
- Created test suite for rate-of-change alerts (5 test cases)
- Created combined alert scenario test
- Fixed rate-of-change detection to use station_code instead of station_id
- All 3 test suites passing (14 total test cases)
Test coverage:
- Zone alerts: P.1 zones 1-8 with INFO/WARNING/CRITICAL/EMERGENCY levels
- Rate-of-change: 0.15/0.25/0.40 m/h thresholds for WARNING/CRITICAL/EMERGENCY
- Combined: Simultaneous zone and rate-of-change alert triggering
Co-Authored-By: Claude <noreply@anthropic.com>
Checkout Action Migration:
- Replace all 'actions/checkout@v4' with 'https://gitea.com/actions/checkout'
- Fixes 'Bad credentials' errors when workflows try to access GitHub API
- Native Gitea checkout action eliminates authentication issues
- Applied across all 4 workflow files (CI, Security, Release, Docs)
Version Increment: 3.1.1 3.1.2
- Core application version updates
- Web API version synchronization
- Documentation version alignment
- Badge and release example updates
Problem Solved:
- Workflows no longer attempt GitHub API calls
- Gitea-native checkout action handles repository access properly
- Eliminates 'Retrieving the default branch name' failures
- Cleaner workflow execution without authentication errors
Files Updated:
- 4 workflow files: checkout action replacement
- 13 files: version number updates
- Consistent v3.1.2 across all components
Benefits:
- Workflows will now run successfully in Gitea
- No more GitHub API authentication failures
- Native Gitea action compatibility
- Ready for successful CI/CD pipeline execution
Features:
- Real-time water level monitoring for Ping River Basin (16 stations)
- Coverage from Chiang Dao to Nakhon Sawan in Northern Thailand
- FastAPI web interface with interactive dashboard and station management
- Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics)
- Comprehensive monitoring with health checks and metrics collection
- Docker deployment with Grafana integration
- Production-ready architecture with enterprise-grade observability
CI/CD & Automation:
- Complete Gitea Actions workflows for CI/CD, security, and releases
- Multi-Python version testing (3.9-3.12)
- Multi-architecture Docker builds (amd64, arm64)
- Daily security scanning and dependency monitoring
- Automated documentation generation
- Performance testing and validation
Production Ready:
- Type safety with Pydantic models and comprehensive type hints
- Data validation layer with range checking and error handling
- Rate limiting and request tracking for API protection
- Enhanced logging with rotation, colors, and performance metrics
- Station management API for dynamic CRUD operations
- Comprehensive documentation and deployment guides
Technical Stack:
- Python 3.9+ with FastAPI and Pydantic
- Multi-database architecture with adapter pattern
- Docker containerization with multi-stage builds
- Grafana dashboards for visualization
- Gitea Actions for CI/CD automation
- Enterprise monitoring and alerting
Ready for deployment to B4L infrastructure!
grabowski
