Northern Thailand Ping River Monitor

Live water levels, discharge, rainfall and machine-learning flood forecasts for the Ping River basin around Chiang Mai. Collects hourly gauge data from public sources, keeps the full history in PostgreSQL, serves a bilingual dashboard and an open REST API, forecasts the flood risk per gauge 6/12/24 hours ahead, and pushes alerts when a gauge crosses its warning or danger level.

Live: water.buildfor.life · API reference at /docs · built by buildfor.life after the October 2024 flood — background in Teaching a Model to See the Ping River Rise 13 Hours Early.

What it does

• Collects hourly water level and discharge from 16 Royal Irrigation Department (RID) telemetry gauges, Chiang Dao to the southern basin, since 2018-08; hourly rainfall and water level from 400+ ThaiWater/HII stations; Open-Meteo catchment rainfall (archive + 48 h forecast); daily state of the large RID dams, Mae Ngat above all. Every source and its quirks: docs/DATA_SOURCES.md.
• Fills gaps. The raw RID grid had readings for ~56 % of hours; a full-history re-fetch plus HII cross-fill brought it to ~93 %. RID's report often arrives hours late in the evening, so late readings are always stored and holes are re-fetched. GET /api/stats reports the current coverage.
• Forecasts. Per gauge and horizon, a gradient-boosted model predicts the rise within 6/12/24 h and the probability of crossing the station's warning and danger levels. Trained on the monitor's own history plus catchment rain; evaluated rolling-origin, event by event. On the October 2024 record flood, trained only on data through August 2024, the first alert came 13 hours before P.1 crossed 3.70 m. Everything about the model, including what did not work: docs/FLOOD_FORECASTING.md.
• Shows it. A Leaflet map with the river drawn as OSM geometry and styled by live discharge, rain gauges, the Chiang Mai inundation zones, per-station history, the forecast card, a replay of the 2024 flood, English/Thai, light/dark.
• Notifies. Public push alerts over a self-hosted ntfy: one message when a gauge crosses its warning or danger level, one all-clear on the way down, an opt-in early-warning topic from the model, nothing in between. Subscribe from the free app, no account. A Matrix room can get the same messages for a team. docs/NOTIFICATIONS.md.

How it runs

 RID gauges · ThaiWater/HII · RID dams · Open-Meteo
                    │ hourly
                    ▼
 ┌──────────────────────────────────────┐        ┌───────────────────────────┐
 │ ping-collector  (Rust, one process)  │        │ monthly retrain (Python)  │
 │  collect → forecast → notify         │◄───────│  scikit-learn, exports    │
 │  website + API on :8000              │ models │  models/flood_*.json.gz   │
 └──────┬──────────────┬────────────────┘        └─────────────┬─────────────┘
        │              │                                       │
        ▼              ▼                                       │
  ntfy topics,    PostgreSQL ◄──────────── history ────────────┘
  Matrix room     (gauges, rain, dams, forecasts, alert state)

• The service is one Rust binary, ping-collector (rust/README.md), run by systemd. It polls RID at :05 past each hour and retries every 10 minutes while the report is behind, fetches HII, the dams and the rain once an hour, forecasts after every cycle that brought new readings, sends the notifications, and serves the dashboard and API, in under 100 MB of memory with every model loaded.
• The models are trained in Python (src/ml, scikit-learn HistGradientBoosting) by a monthly timer and exported for the Rust forecaster, which reproduces Python's features bit for bit and reloads new models without a restart.
• Until October 2026 all of this was a Python/FastAPI app; how and why it moved: docs/RUST_REWRITE.md.

API

Read-only, no key, JSON. Base URL https://water.buildfor.life; timestamps are Asia/Bangkok wall-clock without an offset suffix.

Endpoint	Returns
GET /stations	The 16 RID gauges: code, Thai/English names, coordinates
GET /measurements/latest?limit=N	Newest reading per station
GET /measurements/history/{code}?hours=N	Hourly history; or ?start=YYYY-MM-DD&end=YYYY-MM-DD; limit ≤ 100000
GET /forecast	Current flood-risk forecast, every station × horizon, with thresholds and P.1 inundation-stage probabilities
GET /api/forecast/history/{code}?hours=N&horizon=24	Forecasts as issued, for auditing lead time after the fact
GET /api/forecast/skill?station_code=P.1	Issued forecasts vs what happened, per deployed model version
GET /api/hii/rainfall/latest, /api/hii/waterlevel/latest	Latest ThaiWater/HII gauge readings
GET /api/hii/rainfall/catchment?days=N	HII gauge catchment-mean rain next to the Open-Meteo series the model uses
GET /api/notifications	ntfy server and topic names for the subscribe panel
GET /api/stats	Row counts per source, date range, coverage
GET /health	Database and data freshness (RID, ThaiWater as backup), the RID API, memory
GET /scraping/status	Collection runs, the last forecast run, service version

Interactive reference with schemas: water.buildfor.life/docs (also /redoc, /openapi.json). Responses are cached briefly server-side; poll no faster than once a minute — the data changes hourly. Station editing (POST/PUT/DELETE /stations), /config and POST /scrape/trigger need the server's ADMIN_API_KEY in an X-API-Key header.

Run it locally

Needs Rust >= 1.88 (rustup; on Linux also pkg-config and libssl-dev) and PostgreSQL. Python 3.11 with uv only for training models.

git clone https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor.git
cd Northern-Thailand-Ping-River-Monitor
docker run -d --name ping-pg -e POSTGRES_PASSWORD=ping -e POSTGRES_DB=ping -p 55432:5432 postgres:16
cp .env.example .env
#   POSTGRES_CONNECTION_STRING=postgresql://postgres:ping@localhost:55432/ping
#   WEB_LISTEN=127.0.0.1:8000
cargo build --release --manifest-path rust/Cargo.toml
rust/target/release/ping-collector check-db         # create the tables
rust/target/release/ping-collector fill-gaps 30     # the last 30 days from RID
rust/target/release/ping-collector run              # collect, forecast, serve http://127.0.0.1:8000

Notifications stay off until NTFY_SERVER or MATRIX_* is set. Without trained models the forecast is the persistence heuristic, marked source: "heuristic". Training needs years of history (the server's database, or FLOOD_ML_DB_URL) and the Open-Meteo archive:

uv sync --python 3.11
uv run python scripts/train_flood_model.py --stations all    # writes models/flood_*
uv run python scripts/evaluate_variants.py --stations P.1     # the rolling-origin harness

make help lists the rest (make check runs every CI gate locally).

Deployment

Production is one systemd unit on a small VPS behind Cloudflare; the app directory /opt/thailand-water-monitor is a git checkout. To update:

cd /opt/thailand-water-monitor && git pull
cargo build --release --locked --manifest-path rust/Cargo.toml
sudo bash scripts/install.sh

scripts/install.sh (idempotent, also for a fresh host) installs the binary and ping-collector.service, syncs the Python environment for the monthly water-monitor-retrain.timer, runs a dry run of every source and a forecast check against the Python reference, restarts the service and waits for the website. The retrain trains into models/.staging, refuses to promote anything that is not a rain-enabled (hgb-v3+) set covering the expected stations, and renames the bundles into place; the service picks them up on its next forecast. Details and the operations runbook: docs/FLOOD_FORECASTING.md sections 6–8. Configuration: .env.example.

Repository layout

rust/collector/      ping-collector: collection, forecast run, ntfy/Matrix, website + API (src/web)
rust/forecast/       forecast inference from the exported models; parity tests against Python
src/static/          the dashboard (single file, EN/TH) and its map data
src/data/            the bundled station list
src/ml/              features, training, evaluation harness, export; predict.py is the reference
src/*.py             HII and RID-dam clients/stores for the archive backfills
scripts/             install.sh, retrain.sh, train/evaluate/backfill tools, systemd units
tests/               pytest (synthetic data; database tests need TEST_DATABASE_URL)
docs/                model write-up, data sources, notifications, Matrix, rewrite history
models/              trained bundles (gitignored) and evaluation results

Documentation

• docs/FLOOD_FORECASTING.md — the model: data, features, evaluation, measured performance, negatives, deployment, retraining
• docs/DATA_SOURCES.md — every ingested and candidate source, endpoints, quirks
• docs/NOTIFICATIONS.md — public push alerts: topics, semantics, ntfy deployment
• docs/MATRIX_QUICK_START.md — the same alerts in a Matrix room
• rust/README.md — the service: commands, configuration, develop, deploy
• docs/RUST_REWRITE.md — why and how it moved from Python to Rust
• docs/references/NOTABLE_DOCUMENTS.md — official Thai government resources
• Public overview: buildfor.life/docs/tooling/ping-river-monitor

Contributing

make format before committing, make check must stay green; see CONTRIBUTING.md. Issues and merge requests on git.b4l.co.th.

Data sources and thanks

Royal Irrigation Department (RID) gauge telemetry and reservoir reports; Hydro-Informatics Institute (HII) / ThaiWater open API; Open-Meteo; OpenStreetMap contributors for the river geometry; Chiang Mai Municipality for the inundation map the P.1 stages are keyed to. All instruments are theirs; we aggregate, store, fill gaps and forecast.

License

MIT — see LICENSE.