Northern Thailand Ping River Monitor

Live water levels, discharge, rainfall and machine-learning flood forecasts for the Ping River basin around Chiang Mai. Collects hourly gauge data from public sources, keeps the full history in PostgreSQL, and serves a bilingual dashboard, an open REST API, and 6/12/24-hour flood-risk forecasts per gauge.

Live: water.buildfor.life · API reference at /docs · built by buildfor.life after the October 2024 flood — background in Teaching a Model to See the Ping River Rise 13 Hours Early.

What it does

• Collects hourly water level and discharge from 16 Royal Irrigation Department (RID) telemetry gauges, Chiang Dao to the southern basin, since 2018-08; hourly rainfall and water level from 400+ ThaiWater/HII stations; Open-Meteo catchment rainfall (archive + 48 h forecast); daily Mae Ngat reservoir state. Every source and its quirks: docs/DATA_SOURCES.md.
• Fills gaps. The raw RID grid had readings for ~56 % of hours; a full-history re-fetch plus HII cross-fill brought it to ~93 %. GET /api/stats reports the current figure.
• Forecasts. Per gauge and horizon, a gradient-boosted model predicts the rise within 6/12/24 h and the probability of crossing the station's warning and danger levels. Trained on the monitor's own history plus catchment rain; evaluated rolling-origin, event by event. On the October 2024 record flood, trained only on data through August 2024, the first alert came 13 hours before P.1 crossed 3.70 m. Everything about the model, including what did not work: docs/FLOOD_FORECASTING.md.
• Shows it. A Leaflet map with the river drawn as OSM geometry and styled by live discharge, rain gauges, the Chiang Mai inundation zones, per-station history, the forecast card, a replay of the 2024 flood, English/Thai, light/dark.
• Notifies. Public push alerts over a self-hosted ntfy: one message when a gauge crosses its warning or danger level, one all-clear on the way down, an opt-in early-warning topic from the model, nothing in between. Subscribe from the free app, no account. Matrix room alerts for a team are also supported.

Quick start

Python 3.11 (3.13 breaks the pinned psycopg2-binary), PostgreSQL for anything beyond a quick look, uv.

git clone https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor.git
cd Northern-Thailand-Ping-River-Monitor
uv sync --python 3.11
cp .env.example .env            # DB_TYPE, POSTGRES_CONNECTION_STRING, optional MATRIX_*
uv run python run.py --web-api  # dashboard + API on http://localhost:8000

DB_TYPE=sqlite works for the dashboard and API; the forecasting path expects the PostgreSQL history.

uv run python run.py --status          # collector status
uv run python run.py --test            # one collection cycle
uv run python run.py --fill-gaps 7     # re-fetch the last 7 days from RID
uv run python run.py --collect-hii     # one ThaiWater/HII collection cycle
uv run python run.py --alert-check     # evaluate thresholds, notify Matrix
uv run python scripts/train_flood_model.py --stations all   # retrain (~12 min)
make test                              # pytest, synthetic data, no network
make format                            # black + isort (the CI contract)

RID collection in --web-api mode: polled at :05 past each hour (RID_POLL_OFFSET_MINUTES); while RID has not published the current hour yet, retried every RID_RETRY_MINUTES (10). Every poll stores all readings the report has, and yesterday's report is re-fetched while it still has holes (RID's evening hours often arrive 1-2 h late). On startup the leader re-fetches holes from the last GAP_FILL_DAYS_ON_START (14) days; for older history run --fill-gaps all.

API

Read-only, no key, JSON. Base URL https://water.buildfor.life; timestamps are Asia/Bangkok wall-clock without an offset suffix.

Endpoint	Returns
GET /stations	The 16 RID gauges: code, Thai/English names, coordinates
GET /measurements/latest?limit=N	Newest reading per station
GET /measurements/history/{code}?hours=N	Hourly history; or ?start=YYYY-MM-DD&end=YYYY-MM-DD; limit ≤ 100000
GET /forecast	Current flood-risk forecast, every station × horizon, with thresholds and P.1 inundation-stage probabilities
GET /api/forecast/history/{code}?hours=N&horizon=24	Forecasts as issued, for auditing lead time after the fact
GET /api/hii/rainfall/latest, /api/hii/waterlevel/latest	Latest ThaiWater/HII gauge readings
GET /api/hii/rainfall/catchment?days=N	HII gauge catchment-mean rain next to the Open-Meteo series the model uses
GET /api/forecast/skill?station_code=P.1	Issued forecasts vs what happened, per deployed model version
GET /api/notifications	ntfy server and topic names for the subscribe panel
GET /api/stats	Row counts per source, date range, coverage
GET /health	DB / upstream / memory checks

Interactive reference with schemas: water.buildfor.life/docs. Responses are cached briefly server-side; poll no faster than once a minute — the data changes hourly.

Deployment

Production is a systemd unit on a small VPS behind Cloudflare, updated by git pull. scripts/install.sh (run as root from a checkout) creates the water-monitor user, deploys to /opt/thailand-water-monitor, runs uv sync into .venv, installs water-monitor.service and the monthly water-monitor-retrain.timer.

sudo bash scripts/install.sh
sudo systemctl start water-monitor.service
systemctl list-timers water-monitor-retrain.timer

The retrain timer runs scripts/retrain.sh, which trains into models/.staging, refuses to promote anything that is not a rain-enabled (hgb-v3+) set covering the expected stations, and renames the bundles into place. Details and the operations runbook: docs/FLOOD_FORECASTING.md sections 6–8.

Repository layout

src/                 collector, API (web_api.py), dashboard (static/dashboard.html)
src/ml/              features, training, evaluation harness, prediction, rain/dam/HII loaders
scripts/             train_flood_model.py, retrain.sh, evaluate_variants.py, install.sh, dev_proxy.py
tests/               pytest suite (synthetic data; no DB or network)
docs/                FLOOD_FORECASTING.md, DATA_SOURCES.md, deployment and station guides
models/              trained bundles + metrics.json (gitignored) and evaluation results (tracked)
.gitea/workflows/    ci (format/lint/tests), security (pip-audit/bandit), docs (link + OpenAPI checks)

Documentation

• docs/FLOOD_FORECASTING.md — the model: data, features, evaluation, measured performance, negatives, deployment, retraining
• docs/DATA_SOURCES.md — every ingested and candidate source, endpoints, quirks
• docs/STATION_MANAGEMENT_GUIDE.md — adding/editing gauges
• docs/DATABASE_DEPLOYMENT_GUIDE.md, POSTGRESQL_SETUP.md — database setup
• docs/NOTIFICATIONS.md — public push alerts: topics, semantics, ntfy deployment
• docs/MATRIX_QUICK_START.md — Matrix room alerts for a team
• docs/GAP_FILLING_GUIDE.md — data integrity tooling
• docs/references/NOTABLE_DOCUMENTS.md — official Thai government resources
• Public overview: buildfor.life/docs/tooling/ping-river-monitor

Other database backends (VictoriaMetrics, InfluxDB, MySQL, SQLite) and the Grafana dashboards under grafana/ are supported by the adapters but not what production runs; see docs/VICTORIAMETRICS_SETUP.md if you want them.

Contributing

make format before committing (black 88 columns, isort black profile — the CI gate), make test must stay green, tests use synthetic data only. See CONTRIBUTING.md. Issues and merge requests on git.b4l.co.th.

Data sources and thanks

Royal Irrigation Department (RID) gauge telemetry; Hydro-Informatics Institute (HII) / ThaiWater open API; Open-Meteo; OpenStreetMap contributors for the river geometry; Chiang Mai Municipality for the inundation map the P.1 stages are keyed to. All instruments are theirs; we aggregate, store, fill gaps and forecast.

License

MIT — see LICENSE.