B4L/Northern-Thailand-Ping-River-Monitor
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 15 Wiki Activity

docs: README describes the project that exists; CLAUDE.md for agents

Browse Source 
The README was the original template: P.1 "in Nakhon Sawan", P.103 "in
Bangkok", VictoriaMetrics as the recommended database, Docker/Grafana
sections, github.com/your-username support links. Rewritten around what
runs: sources, gap fill, the forecast and its 13 h result, the live API
table, systemd deployment with the retrain timer, repository layout,
real docs links, data-source credits. Other DB adapters are mentioned as
supported-but-not-production.

CLAUDE.md replaces the untracked Ruflo boilerplate with project rules:
Python 3.11, format/test gates, Bangkok timestamps, harness-first model
changes judged on lead, the RainUnavailableError guard, no git add -A.
This commit is contained in:
grabowski
2026-09-11 23:44:43 +02:00
parent b03318210c
commit 2e19974fad
2 changed files with 154 additions and 458 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+40
View File
@@ -0,0 +1,40 @@
# CLAUDE.md
Guidance for AI coding agents working in this repository.
## What this is
Flood monitoring and forecasting for the Ping River, Chiang Mai. Public dashboard and
API at https://water.buildfor.life/ (never publish the server's private/Tailscale IP).
Production: one systemd unit on a small VPS, `/opt/thailand-water-monitor`, user
`water-monitor`, interpreter `.venv/bin/python` (uv-managed), updated by `git pull`.
## Rules
- Python 3.11 only. `uv sync --python 3.11`; run everything as `uv run ...`.
- `make format` (black 88 / isort black profile, config in pyproject.toml) before
committing; CI fails on formatting. `make test` must stay green — tests are
synthetic-data only, never add one that needs the DB or network.
- Timestamps everywhere are Asia/Bangkok wall-clock with no offset. The dashboard
parses them with `parseTs()` and renders with `timeZone: TZ`; keep it that way.
- Model changes go through the rolling-origin harness (`scripts/evaluate_variants.py`)
and are judged on first-alert LEAD and false alarms, not MAE. Record results, positive
or negative, in `docs/FLOOD_FORECASTING.md` section 5. Do not change what is deployed
(`rise_rain` / hgb-v3) without a harness result that beats it on lead.
- `train_all()` must never silently produce a gauge-only (v2) model; the guard that
raises `RainUnavailableError` stays.
- No `git add -A`: zero-byte shell-accident files (`#`, `$(wc`, ...) have been committed
before. Stage files by name.
- Do not add Co-Authored-By trailers.
- The dashboard is a single file, `src/static/dashboard.html`, EN + TH via the `t()`
table: every user-visible string needs both languages.
## Where things are
- `src/web_api.py` FastAPI app; `src/water_scraper_v3.py` RID collector;
`src/hii_collector.py` ThaiWater/HII; `src/ml/` features/train/evaluate/predict,
`rain.py` (Open-Meteo), `dam.py`, `hii_rain.py`.
- `scripts/retrain.sh` + `water-monitor-retrain.timer`: monthly retrain with staged
promote. `scripts/dev_proxy.py`: serve the working-copy dashboard against the live API.
- `docs/FLOOD_FORECASTING.md` is the authoritative model write-up; `docs/DATA_SOURCES.md`
the source catalog.
README.md
+114 -458
View File
@@ -1,494 +1,150 @@
# Northern Thailand Ping River Monitor 🏔️
# Northern Thailand Ping River Monitor
A comprehensive real-time water level monitoring system for the Ping River Basin in Northern Thailand, covering Royal Irrigation Department (RID) stations from Chiang Dao to Nakhon Sawan with advanced data collection, storage, and visualization capabilities.
Live water levels, discharge, rainfall and machine-learning flood forecasts for the
Ping River basin around Chiang Mai. Collects hourly gauge data from public sources,
keeps the full history in PostgreSQL, and serves a bilingual dashboard, an open REST
API, and 6/12/24-hour flood-risk forecasts per gauge.
**Live dashboard: [water.buildfor.life](https://water.buildfor.life/)** — water levels, discharge, rainfall and 6/12/24 h flood forecasts for Chiang Mai, in English and Thai. Background: [Teaching a Model to See the Ping River Rise 13 Hours Early](https://buildfor.life/blog/ping-river-monitor/).
**Live: [water.buildfor.life](https://water.buildfor.life/)** · API reference at
[/docs](https://water.buildfor.life/docs) · built by [buildfor.life](https://buildfor.life)
after the [October 2024 flood](https://buildfor.life/blog/chiang-mai-flood-2024/) —
background in [Teaching a Model to See the Ping River Rise 13 Hours Early](https://buildfor.life/blog/ping-river-monitor/).
[![CI/CD](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/ci.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions) [![Security](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/security.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions) [![Documentation](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/docs.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions) [![Python](https://img.shields.io/badge/Python-3.11-blue.svg)](https://python.org) [![FastAPI](https://img.shields.io/badge/FastAPI-0.104+-green.svg)](https://fastapi.tiangolo.com) [![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://docker.com) [![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE) [![Version](https://img.shields.io/badge/Version-v3.1.3-blue.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/releases)
[![CI](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/ci.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
[![Security](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/security.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
[![Docs](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/docs.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
[![Python 3.11](https://img.shields.io/badge/Python-3.11-blue.svg)](https://python.org)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
## 🌟 Features
## What it does
### 📊 **Real-time Data Collection**
- **16 Monitoring Stations** across Thailand
- **15-minute Collection Frequency** with intelligent scheduling
- **Automatic Gap Filling** for missing historical data
- **Data Validation** and error recovery mechanisms
- **Rate Limiting** to prevent API abuse
- **Collects** hourly water level and discharge from 16 Royal Irrigation Department
(RID) telemetry gauges, Chiang Dao to the southern basin, since 2018-08; hourly
rainfall and water level from 400+ ThaiWater/HII stations; Open-Meteo catchment
rainfall (archive + 48 h forecast); daily Mae Ngat reservoir state. Every source and
its quirks: [docs/DATA_SOURCES.md](docs/DATA_SOURCES.md).
- **Fills gaps.** The raw RID grid had readings for ~56 % of hours; a full-history
re-fetch plus HII cross-fill brought it to ~93 %. `GET /api/stats` reports the
current figure.
- **Forecasts.** Per gauge and horizon, a gradient-boosted model predicts the rise
within 6/12/24 h and the probability of crossing the station's warning and danger
levels. Trained on the monitor's own history plus catchment rain; evaluated
rolling-origin, event by event. On the October 2024 record flood, trained only on
data through August 2024, the first alert came **13 hours before** P.1 crossed
3.70 m. Everything about the model, including what did not work:
[docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md).
- **Shows it.** A Leaflet map with the river drawn as OSM geometry and styled by live
discharge, rain gauges, the Chiang Mai inundation zones, per-station history, the
forecast card, a replay of the 2024 flood, English/Thai, light/dark.
- **Alerts** (optional) to a Matrix room when a gauge crosses its thresholds.
### 🌐 **Web API Interface (NEW!)**
- **FastAPI-powered REST API** with interactive documentation
- **Station Management** - Add, update, and remove monitoring stations
- **Real-time health monitoring** and system status
- **Manual data collection triggers** via web interface
- **Comprehensive metrics** and performance monitoring
- **CORS support** for web applications
## Quick start
### 🗄️ **Multi-Database Support**
- **VictoriaMetrics** (Recommended) - High-performance time-series
- **InfluxDB** - Purpose-built time-series database
- **PostgreSQL + TimescaleDB** - Relational with time-series optimization
- **MySQL** - Traditional relational database
- **SQLite** - Local development and testing
### 🗺️ **Geolocation Support**
- **Grafana Geomap** integration ready
- **GPS coordinates** and geohash support
- **Interactive mapping** of water stations
### 📈 **Visualization & Monitoring**
- **Pre-built Grafana dashboards**
- **Real-time alerts** and notifications
- **Historical trend analysis**
- **Built-in metrics collection** (counters, gauges, histograms)
- **Health checks** for database, API, and system resources
### 🚀 **Production Ready**
- **Docker containerization** with multi-service support
- **Systemd service** configuration
- **HTTPS support** with SSL certificates
- **Comprehensive logging** with rotation and colored output
- **Type safety** with Pydantic models and type hints
- **Custom exception handling** for better error management
## 🚀 Quick Start
### Prerequisites
- Python 3.9 or higher
- Internet connection for data fetching
- Database server (optional - SQLite works out of the box)
### Installation
Python **3.11** (3.13 breaks the pinned `psycopg2-binary`), PostgreSQL for anything
beyond a quick look, [uv](https://docs.astral.sh/uv/).
```bash
# Clone the repository
git clone https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor.git
cd Northern-Thailand-Ping-River-Monitor
# Quick setup with Make
make dev-setup
# Or manual setup:
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env
uv sync --python 3.11
cp .env.example .env # DB_TYPE, POSTGRES_CONNECTION_STRING, optional MATRIX_*
uv run python run.py --web-api # dashboard + API on http://localhost:8000
```
### Basic Usage
`DB_TYPE=sqlite` works for the dashboard and API; the forecasting path expects the
PostgreSQL history.
```bash
# Test run with SQLite (default)
make run-test
# or: python run.py --test
# Run continuous monitoring
make run
# or: python run.py
# Start web API server (NEW!)
make run-api
# or: python run.py --web-api
# Run all tests
make test
# Demo different databases
python src/demo_databases.py
uv run python run.py --status # collector status
uv run python run.py --test # one collection cycle
uv run python run.py --fill-gaps 7 # re-fetch the last 7 days from RID
uv run python run.py --collect-hii # one ThaiWater/HII collection cycle
uv run python run.py --alert-check # evaluate thresholds, notify Matrix
uv run python scripts/train_flood_model.py --stations all # retrain (~12 min)
make test # pytest, synthetic data, no network
make format # black + isort (the CI contract)
```
### 🌐 Web API Interface (NEW!)
## API
The system now includes a comprehensive FastAPI web interface:
Read-only, no key, JSON. Base URL `https://water.buildfor.life`; timestamps are
Asia/Bangkok wall-clock without an offset suffix.
| Endpoint | Returns |
| --- | --- |
| `GET /stations` | The 16 RID gauges: code, Thai/English names, coordinates |
| `GET /measurements/latest?limit=N` | Newest reading per station |
| `GET /measurements/history/{code}?hours=N` | Hourly history; or `?start=YYYY-MM-DD&end=YYYY-MM-DD`; `limit` ≤ 100000 |
| `GET /forecast` | Current flood-risk forecast, every station × horizon, with thresholds and P.1 inundation-stage probabilities |
| `GET /api/forecast/history/{code}?hours=N&horizon=24` | Forecasts as issued, for auditing lead time after the fact |
| `GET /api/hii/rainfall/latest`, `/api/hii/waterlevel/latest` | Latest ThaiWater/HII gauge readings |
| `GET /api/hii/rainfall/catchment?days=N` | HII gauge catchment-mean rain next to the Open-Meteo series the model uses |
| `GET /api/stats` | Row counts per source, date range, coverage |
| `GET /health` | DB / upstream / memory checks |
Interactive reference with schemas: [water.buildfor.life/docs](https://water.buildfor.life/docs).
Responses are cached briefly server-side; poll no faster than once a minute — the data
changes hourly.
## Deployment
Production is a systemd unit on a small VPS behind Cloudflare, updated by `git pull`.
`scripts/install.sh` (run as root from a checkout) creates the `water-monitor` user,
deploys to `/opt/thailand-water-monitor`, runs `uv sync` into `.venv`, installs
`water-monitor.service` and the monthly `water-monitor-retrain.timer`.
```bash
# Start the web API
python run.py --web-api
# Access the API at:
# - Dashboard: http://localhost:8000
# - Interactive docs: http://localhost:8000/docs
# - Health check: http://localhost:8000/health
# - Latest data: http://localhost:8000/measurements/latest
```
**Key API Endpoints:**
- `GET /` - Web dashboard
- `GET /health` - System health status
- `GET /metrics` - Application metrics
- `GET /stations` - List all monitoring stations
- `POST /stations` - Add new monitoring station
- `PUT /stations/{id}` - Update station information
- `DELETE /stations/{id}` - Remove monitoring station
- `GET /measurements/latest` - Latest measurements
- `GET /measurements/station/{code}` - Station-specific data
- `POST /scrape/trigger` - Trigger manual data collection
## 📊 Station Information
The system monitors **16 water stations** along the Ping River Basin in Northern Thailand:
| Station | Thai Name | English Name | Location |
|---------|-----------|--------------|----------|
| P.1 | สะพานนวรัฐ | Nawarat Bridge | Nakhon Sawan |
| P.5 | สะพานท่านาง | Tha Nang Bridge | - |
| P.20 | บ้านเชียงดาว | Ban Chiang Dao | Chiang Mai |
| P.21 | บ้านริมใต้ | Ban Rim Tai | - |
| P.4A | บ้านแม่แตง | Ban Mae Taeng | Chiang Mai |
| P.67 | บ้านแม่แต | Ban Tae | - |
| P.75 | บ้านช่อแล | Ban Chai Lat | - |
| P.76 | บ้านแม่อีไฮ | Banb Mae I Hai | - |
| P.77 | บ้านสบแม่สะป๊วด | Baan Sop Mae Sapuord | - |
| P.81 | บ้านโป่ง | Ban Pong | - |
| P.82 | บ้านสบวิน | Ban Sob win | - |
| P.84 | บ้านพันตน | Ban Panton | - |
| P.85 | บ้านหล่ายแก้ว | Baan Lai Kaew | - |
| P.87 | บ้านป่าซาง | Ban Pa Sang | - |
| P.92 | บ้านเมืองกึ๊ด | Ban Muang Aut | - |
| P.103 | สะพานวงแหวนรอบ 3 | Ring Bridge 3 | Bangkok |
### Data Metrics
- **Water Level**: Measured in meters (m)
- **Discharge**: Flow rate in cubic meters per second (cms)
- **Discharge Percentage**: Relative to station capacity
- **Timestamp**: Thai time (UTC+7) with Buddhist calendar support
## 🗄️ Database Configuration
### VictoriaMetrics (Recommended)
**High-performance time-series database with excellent compression and query speed.**
```bash
# Environment variables
export DB_TYPE=victoriametrics
export VM_HOST=localhost
export VM_PORT=8428
# Quick start with Docker
docker run -d \
--name victoriametrics \
-p 8428:8428 \
-v victoria-metrics-data:/victoria-metrics-data \
victoriametrics/victoria-metrics:latest \
--storageDataPath=/victoria-metrics-data \
--retentionPeriod=2y \
--httpListenAddr=:8428
```
### Complete Stack with Grafana
```bash
# Start the complete monitoring stack
docker-compose -f docker-compose.victoriametrics.yml up -d
# Access Grafana at http://localhost:3000
# Username: admin, Password: admin_password
```
### Other Database Options
<details>
<summary>InfluxDB Configuration</summary>
```bash
export DB_TYPE=influxdb
export INFLUX_HOST=localhost
export INFLUX_PORT=8086
export INFLUX_DATABASE=water_monitoring
export INFLUX_USERNAME=water_user
export INFLUX_PASSWORD=your_password
```
</details>
<details>
<summary>PostgreSQL Configuration</summary>
```bash
export DB_TYPE=postgresql
export POSTGRES_CONNECTION_STRING=postgresql://user:password@localhost:5432/water_monitoring
```
</details>
<details>
<summary>MySQL Configuration</summary>
```bash
export DB_TYPE=mysql
export MYSQL_CONNECTION_STRING=mysql://user:password@localhost:3306/water_monitoring
```
</details>
## 📈 Grafana Dashboards
### Pre-built Dashboard Features
- **Real-time water levels** across all stations
- **Historical trends** and patterns
- **Discharge monitoring** with percentage indicators
- **Station status** and health monitoring
- **Geomap visualization** of station locations
- **Alert thresholds** for critical water levels
### Sample Queries
**VictoriaMetrics/Prometheus:**
```promql
# Current water levels
water_level
# High discharge alerts
water_discharge_percent > 80
# Station-specific data
water_level{station_code="P.1"}
```
**SQL Databases:**
```sql
-- Latest readings from all stations
SELECT s.station_code, s.english_name, m.water_level, m.discharge
FROM stations s
JOIN water_measurements m ON s.id = m.station_id
WHERE m.timestamp = (SELECT MAX(timestamp) FROM water_measurements WHERE station_id = s.id);
```
## 🚀 Production Deployment
### Docker Deployment
```bash
# Build the image
docker build -t thailand-water-monitor .
# Run with environment variables
docker run -d \
--name water-monitor \
-e DB_TYPE=victoriametrics \
-e VM_HOST=victoriametrics \
thailand-water-monitor
```
### Systemd Service (Linux)
The install script sets everything up: a dedicated `water-monitor` system user,
a deploy to `/opt/thailand-water-monitor`, a uv-managed virtualenv, and the
enabled systemd unit.
```bash
# From a checkout of the repo, as root:
sudo bash scripts/install.sh
# Then start and check:
sudo systemctl start water-monitor.service
systemctl status water-monitor.service
systemctl list-timers water-monitor-retrain.timer
```
Fill in `/opt/thailand-water-monitor/.env` (Matrix token/room, DB settings)
before starting if the script reports it is missing.
The retrain timer runs `scripts/retrain.sh`, which trains into `models/.staging`,
refuses to promote anything that is not a rain-enabled (`hgb-v3+`) set covering the
expected stations, and renames the bundles into place. Details and the operations
runbook: [docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md) sections 6–8.
<details>
<summary>Manual setup (if you prefer not to use the script)</summary>
```bash
sudo useradd --system --no-create-home --shell /usr/sbin/nologin water-monitor
uv sync --python 3.11 # creates .venv, the interpreter both units run
sudo cp scripts/water-monitor.service scripts/water-monitor-retrain.service scripts/water-monitor-retrain.timer /etc/systemd/system/
sudo systemctl enable --now water-monitor.service water-monitor-retrain.timer
```
</details>
## 🔧 Command Line Tools
### Main Application
```bash
python src/water_scraper_v3.py # Run continuous monitoring
python src/water_scraper_v3.py --test # Single test cycle
python src/water_scraper_v3.py --help # Show help
```
### Data Management
```bash
python src/water_scraper_v3.py --check-gaps 7 # Check for missing data (7 days)
python src/water_scraper_v3.py --fill-gaps 7 # Fill missing data gaps
python src/water_scraper_v3.py --update-data 2 # Update existing data (2 days)
```
### Database Testing
```bash
python src/demo_databases.py # SQLite demo
python src/demo_databases.py victoriametrics # VictoriaMetrics demo
python src/demo_databases.py all # Test all databases
```
## 📚 Documentation
### Core Documentation
- **[Data Sources & API Catalog](docs/DATA_SOURCES.md)** - Every ingested and available data source (RID, ThaiWater/HII, dams, rainfall, forecasts)
- **[Installation Guide](docs/DATABASE_DEPLOYMENT_GUIDE.md)** - Complete setup instructions
- **[Gap Filling Guide](docs/GAP_FILLING_GUIDE.md)** - Data integrity management
### Deployment Guides
- **[VictoriaMetrics Setup](docs/VICTORIAMETRICS_SETUP.md)** - High-performance deployment
- **[Debian Troubleshooting](docs/DEBIAN_TROUBLESHOOTING.md)** - Linux deployment issues
### References
- **[Notable Documents](docs/references/NOTABLE_DOCUMENTS.md)** - Official Thai government resources
## 🔍 Troubleshooting
### Common Issues
**Database Connection Errors:**
```bash
# Check database status
python src/demo_databases.py
# Test specific database
python src/demo_databases.py victoriametrics
```
**Missing Data:**
```bash
# Check for gaps
python src/water_scraper_v3.py --check-gaps 7
# Fill missing data
python src/water_scraper_v3.py --fill-gaps 7
```
**Service Issues:**
```bash
# Check service status
sudo systemctl status water-monitor
# View logs
sudo journalctl -u water-monitor -f
```
### Health Checks
```bash
# VictoriaMetrics health
curl http://localhost:8428/health
# Check latest data
curl "http://localhost:8428/api/v1/query?query=water_level"
# Application logs
tail -f water_monitor.log
```
## 🌐 API Integration
### VictoriaMetrics API Examples
```bash
# Query current water levels
curl "http://localhost:8428/api/v1/query?query=water_level"
# Query discharge rates for last hour
curl "http://localhost:8428/api/v1/query_range?query=water_discharge&start=$(date -d '1 hour ago' +%s)&end=$(date +%s)&step=300"
# Query specific station
curl "http://localhost:8428/api/v1/query?query=water_level{station_code=\"P.1\"}"
# High discharge alerts
curl "http://localhost:8428/api/v1/query?query=water_discharge_percent>80"
```
## 📊 Performance
### System Requirements
- **CPU**: 1-2 cores (minimal load)
- **RAM**: 512MB - 2GB (depending on database)
- **Storage**: 1GB+ (for historical data)
- **Network**: Stable internet connection
### Performance Metrics
- **Data Collection**: ~300 data points every 15 minutes
- **Database Write Speed**: 1000+ points/second (VictoriaMetrics)
- **Query Response**: <100ms for recent data
- **Storage Efficiency**: 70x compression vs. raw data
## 🤝 Contributing
Contributions are welcome! Please:
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request
### Development Setup
```bash
# Clone your fork
git clone https://github.com/your-username/thailand-water-monitor.git
cd thailand-water-monitor
# Install development dependencies
pip install -r requirements.txt
pip install pytest black flake8
# Run tests
pytest
# Format code
black src/
```
## 📄 License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## 🙏 Acknowledgments
- **Royal Irrigation Department (RID)** of Thailand for providing the data API
- **VictoriaMetrics** team for the excellent time-series database
- **Grafana** team for the visualization platform
- **Python community** for the amazing libraries and tools
## 📞 Support
- **Issues**: [GitHub Issues](https://github.com/your-username/thailand-water-monitor/issues)
- **Discussions**: [GitHub Discussions](https://github.com/your-username/thailand-water-monitor/discussions)
- **Documentation**: [Project Wiki](https://github.com/your-username/thailand-water-monitor/wiki)
---
## 📁 Project Structure
## Repository layout
```
Northern-Thailand-Ping-River-Monitor/
├── src/ # Main application code
├── tests/ # Test suite
├── docs/ # Documentation
├── grafana/ # Grafana dashboards
├── scripts/ # Utility scripts
├── docker-compose.yml # Docker deployment
├── Makefile # Development tasks
└── requirements.txt # Dependencies
src/ collector, API (web_api.py), dashboard (static/dashboard.html)
src/ml/ features, training, evaluation harness, prediction, rain/dam/HII loaders
scripts/ train_flood_model.py, retrain.sh, evaluate_variants.py, install.sh, dev_proxy.py
tests/ pytest suite (synthetic data; no DB or network)
docs/ FLOOD_FORECASTING.md, DATA_SOURCES.md, deployment and station guides
models/ trained bundles + metrics.json (gitignored) and evaluation results (tracked)
.gitea/workflows/ ci (format/lint/tests), security (pip-audit/bandit), docs (link + OpenAPI checks)
```
See [docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md) for the forecasting architecture and [docs/DATA_SOURCES.md](docs/DATA_SOURCES.md) for the data pipeline.
## Documentation
## 🔄 CI/CD & Automation
- [docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md) — the model: data, features, evaluation, measured performance, negatives, deployment, retraining
- [docs/DATA_SOURCES.md](docs/DATA_SOURCES.md) — every ingested and candidate source, endpoints, quirks
- [docs/STATION_MANAGEMENT_GUIDE.md](docs/STATION_MANAGEMENT_GUIDE.md) — adding/editing gauges
- [docs/DATABASE_DEPLOYMENT_GUIDE.md](docs/DATABASE_DEPLOYMENT_GUIDE.md), [POSTGRESQL_SETUP.md](POSTGRESQL_SETUP.md) — database setup
- [docs/MATRIX_QUICK_START.md](docs/MATRIX_QUICK_START.md) — alert delivery
- [docs/GAP_FILLING_GUIDE.md](docs/GAP_FILLING_GUIDE.md) — data integrity tooling
- [docs/references/NOTABLE_DOCUMENTS.md](docs/references/NOTABLE_DOCUMENTS.md) — official Thai government resources
- Public overview: [buildfor.life/docs/tooling/ping-river-monitor](https://buildfor.life/docs/tooling/ping-river-monitor/)
The project includes comprehensive Gitea Actions workflows:
Other database backends (VictoriaMetrics, InfluxDB, MySQL, SQLite) and the Grafana
dashboards under `grafana/` are supported by the adapters but not what production
runs; see [docs/VICTORIAMETRICS_SETUP.md](docs/VICTORIAMETRICS_SETUP.md) if you want them.
- **🧪 CI/CD Pipeline** - Automated testing, building, and deployment
- **🔒 Security Scanning** - Daily vulnerability and dependency checks
- **📚 Documentation** - Automated API docs and validation
- **🚀 Release Management** - Automated releases with multi-arch Docker builds
## Contributing
See [docs/GITEA_WORKFLOWS.md](docs/GITEA_WORKFLOWS.md) for detailed workflow documentation.
`make format` before committing (black 88 columns, isort black profile — the CI gate),
`make test` must stay green, tests use synthetic data only. See
[CONTRIBUTING.md](CONTRIBUTING.md). Issues and merge requests on
[git.b4l.co.th](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor).
## 🔗 Repository
## Data sources and thanks
- **Main Repository**: https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor
- **Issues**: https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/issues
- **Actions**: https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions
- **Documentation**: [docs/](docs/)
Royal Irrigation Department (RID) gauge telemetry; Hydro-Informatics Institute (HII) /
ThaiWater open API; Open-Meteo; OpenStreetMap contributors for the river geometry;
Chiang Mai Municipality for the inundation map the P.1 stages are keyed to. All
instruments are theirs; we aggregate, store, fill gaps and forecast.
**Made with ❤️ for water resource monitoring in Northern Thailand's Ping River Basin**
## License
MIT — see [LICENSE](LICENSE).