Commit Graph
2 Commits
Author SHA1 Message Date
grabowski 4358d52d55 feat: ML flood-event forecasting from 8 years of gauge history
Security & Dependency Updates / Dependency Security Scan (push) Successful in 1m8s
Security & Dependency Updates / License Compliance (push) Successful in 25s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 20s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 17s
Security & Dependency Updates / Security Summary (push) Successful in 9s
Add src/ml/ package predicting, per station and per 6/12/24 h horizon,
the probability of exceeding warning (3.0 m) and danger (4.5 m) levels
plus expected peak level, trained on the 592k-row PostgreSQL history:

- features.py: hourly grid with coverage gating and no future leakage;
  upstream stations enter at empirically measured travel-time lags
  (P.20 +17h ... P.103 +1h vs P.1); hour-of-day deliberately excluded
  (it encodes the scrape schedule, not hydrology)
- train.py: HistGradientBoosting regression + warn/danger classifier
  heads per station x horizon, >=30-positives gate with calibrated
  sigmoid-on-regression fallback, strict temporal splits, per-event
  lead-time evaluation; guards against sklearn 1.9.0 crash on
  degenerate feature columns
- predict.py: bundle loading with feature-name checks, heuristic
  fallback tier, get_latest_forecasts() for the API; raises when no
  models are trained so the endpoint 503s instead of serving
  persistence output as forecasts
- data.py: Postgres-first loader (FLOOD_ML_DB_URL override), HTTP API
  fallback (flagged: that path backfills synthetic discharge), csv.gz
  cache
- /forecast endpoint (15-min TTL cache) + dashboard flood-risk panel
  (hidden until models exist)
- docs/FLOOD_FORECASTING.md: full system doc with measured deployment
  numbers (~335 MB RSS, CPU negligible, ~6 min full retrain) and
  retraining policy

Validation: out-of-sample backtest of the record 2024 flood season
(train <= Aug 2024) alerted 24-48 h ahead of the Oct 5 peak; 2025-26
test split: P.1 6h PR-AUC 0.974, recall 98.3% at 1% false-alarm rate.

Also: fix P.81 station coordinates (was Ban Pong/Ratchaburi, 493 km
out of basin; now 18.6936 N 99.0819 E per RID station page), pin
scikit-learn==1.9.0 and numpy<2, gitignore model artifacts (~100 MB,
train on the server via scripts/train_flood_model.py).
2026-08-10 12:49:47 +07:00
grabowski af62cfef0b Initial commit: Northern Thailand Ping River Monitor v3.1.0
Security & Dependency Updates / Dependency Security Scan (push) Successful in 29s
Security & Dependency Updates / Docker Security Scan (push) Failing after 53s
Security & Dependency Updates / License Compliance (push) Successful in 13s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 19s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 11s
Security & Dependency Updates / Security Summary (push) Successful in 7s
Features:
- Real-time water level monitoring for Ping River Basin (16 stations)
- Coverage from Chiang Dao to Nakhon Sawan in Northern Thailand
- FastAPI web interface with interactive dashboard and station management
- Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics)
- Comprehensive monitoring with health checks and metrics collection
- Docker deployment with Grafana integration
- Production-ready architecture with enterprise-grade observability

 CI/CD & Automation:
- Complete Gitea Actions workflows for CI/CD, security, and releases
- Multi-Python version testing (3.9-3.12)
- Multi-architecture Docker builds (amd64, arm64)
- Daily security scanning and dependency monitoring
- Automated documentation generation
- Performance testing and validation

 Production Ready:
- Type safety with Pydantic models and comprehensive type hints
- Data validation layer with range checking and error handling
- Rate limiting and request tracking for API protection
- Enhanced logging with rotation, colors, and performance metrics
- Station management API for dynamic CRUD operations
- Comprehensive documentation and deployment guides

 Technical Stack:
- Python 3.9+ with FastAPI and Pydantic
- Multi-database architecture with adapter pattern
- Docker containerization with multi-stage builds
- Grafana dashboards for visualization
- Gitea Actions for CI/CD automation
- Enterprise monitoring and alerting

 Ready for deployment to B4L infrastructure!
2025-08-12 15:40:24 +07:00