grabowski
701bb87d34
CMU Water Center's Chiang Mai flood hazard map (CMFlood) replaces the older municipal map. It is built on the October 2024 flood and has the Ping leave its banks above P.1 4.20 m, with five stages: 4.30, 4.50, 4.70, 5.00 and 5.30 m (the 2024 record). The old map's seven stages (3.70-4.60 m) said the city floods from 3.70 m; CMU calls that map out of date. - P1_FLOOD_STAGES: the five CMU stages, discharge on RID's 2024 rating. /forecast's P.1 `stages` now has five entries (same format). Thresholds are unchanged: 4.20 m danger is the bank top, 3.70 m warning half a metre below (as on CMU's gauge page). flood_config.json and the Rust parity fixtures regenerated (only the stages changed). - flood-zones-p1.geojson: CMU's KML (L1..L5) as five disjoint zones, zone k floods first at stage k; scripts/build_flood_zones.py rebuilds it. - Dashboard: CMU's colours and credit (map attribution); the stage grid follows the number of stages; zone lookup for markers honours holes. The city verdict uses statusOf's classes: flooding (over the banks) from 4.20 m, high water from 3.70 m. Zones show by themselves when the river is over its banks or stage 1 is >= 50 % likely. Alert, replay and verdict texts no longer call 3.70 m city flooding (EN + TH). - Docs: DATA_SOURCES 5.1 (CMFlood: the map, flood marks, poles, city gauges), FLOOD_FORECASTING, DASHBOARD_REDESIGN, NOTIFICATIONS, README credits.
12 KiB
Northern Thailand Ping River Monitor
Live water levels, discharge, rainfall and machine-learning flood forecasts for the Ping River basin around Chiang Mai. Collects hourly gauge data from public sources, keeps the full history in PostgreSQL, serves a bilingual dashboard and an open REST API, forecasts the flood risk per gauge 6/12/24 hours ahead, and pushes alerts when a gauge crosses its warning or danger level.
Live: water.buildfor.life · API reference at /docs · built by buildfor.life after the October 2024 flood — background in Teaching a Model to See the Ping River Rise 13 Hours Early.
What it does
- Collects hourly water level and discharge from 16 Royal Irrigation Department (RID) telemetry gauges, Chiang Dao to the southern basin, since 2018-08; hourly rainfall and water level from 400+ ThaiWater/HII stations; Open-Meteo catchment rainfall (archive + 48 h forecast); daily state of the large RID dams, Mae Ngat above all. Every source and its quirks: docs/DATA_SOURCES.md.
- Fills gaps. The raw RID grid had readings for ~56 % of hours; a full-history
re-fetch plus HII cross-fill brought it to ~93 %. RID's report often arrives
hours late in the evening, so late readings are always stored and holes are
re-fetched.
GET /api/statsreports the current coverage. - Forecasts. Per gauge and horizon, a gradient-boosted model predicts the rise within 6/12/24 h and the probability of crossing the station's warning and danger levels. Trained on the monitor's own history plus catchment rain; evaluated rolling-origin, event by event. On the October 2024 record flood, trained only on data through August 2024, the first alert came 13 hours before P.1 crossed 3.70 m. Everything about the model, including what did not work: docs/FLOOD_FORECASTING.md.
- Shows it. One screen: Chiang Mai's flood verdict, the gauges with their status and trend, a self-hosted map of the basin with rain and the city's inundation zones, and a panel per station with its history since 2018, the forecast and how accurate it has been. ThaiWater's basin stations, a replay of the 2024 flood, Thai/English, dark/light.
- Notifies. Public push alerts over a self-hosted ntfy: one message when a gauge crosses its warning or danger level, one all-clear on the way down, an opt-in early-warning topic from the model, nothing in between. Subscribe from the free app, no account. A Matrix room can get the same messages for a team. docs/NOTIFICATIONS.md.
How it runs
RID gauges · ThaiWater/HII · RID dams · Open-Meteo
│ hourly
▼
┌──────────────────────────────────────┐ ┌───────────────────────────┐
│ ping-collector (Rust, one process) │ │ monthly retrain (Python) │
│ collect → forecast → notify │◄───────│ scikit-learn, exports │
│ website + API on :8000 │ models │ models/flood_*.json.gz │
└──────┬──────────────┬────────────────┘ └─────────────┬─────────────┘
│ │ │
▼ ▼ │
ntfy topics, PostgreSQL ◄──────────── history ────────────┘
Matrix room (gauges, rain, dams, forecasts, alert state)
- The service is one Rust binary,
ping-collector(rust/README.md), run by systemd. It polls RID at :05 past each hour and retries every 10 minutes while the report is behind, fetches HII, the dams and the rain once an hour, forecasts after every cycle that brought new readings, sends the notifications, and serves the dashboard and API, in under 100 MB of memory with every model loaded. - The models are trained in Python (
src/ml, scikit-learn HistGradientBoosting) by a monthly timer and exported for the Rust forecaster, which reproduces Python's features bit for bit and reloads new models without a restart. - Until October 2026 all of this was a Python/FastAPI app; how and why it moved: docs/RUST_REWRITE.md.
API
Read-only, no key, JSON. Base URL https://water.buildfor.life; timestamps are
Asia/Bangkok wall-clock without an offset suffix.
| Endpoint | Returns |
|---|---|
GET /stations |
The 16 RID gauges: code, Thai/English names, coordinates |
GET /measurements/latest?limit=N |
Newest reading per station |
GET /measurements/history/{code}?hours=N |
Hourly history; or ?start=YYYY-MM-DD&end=YYYY-MM-DD; limit ≤ 100000 |
GET /forecast |
Current flood-risk forecast, every station × horizon, with thresholds and P.1 inundation-stage probabilities |
GET /api/forecast/history/{code}?hours=N&horizon=24 |
Forecasts as issued, for auditing lead time after the fact |
GET /api/forecast/skill?station_code=P.1 |
Issued forecasts vs what happened, per deployed model version |
GET /api/hii/rainfall/latest, /api/hii/waterlevel/latest |
Latest ThaiWater/HII gauge readings |
GET /api/hii/rainfall/catchment?days=N |
HII gauge catchment-mean rain next to the Open-Meteo series the model uses |
GET /api/notifications |
ntfy server and topic names for the subscribe panel |
GET /api/stats |
Row counts per source, date range, coverage |
GET /health |
Database and data freshness (RID, ThaiWater as backup), the RID API, memory |
GET /scraping/status |
Collection runs, the last forecast run, service version |
Interactive reference with schemas: water.buildfor.life/docs
(also /redoc, /openapi.json). Responses are cached briefly server-side; poll no
faster than once a minute — the data changes hourly. Station editing
(POST/PUT/DELETE /stations), /config and POST /scrape/trigger need the
server's ADMIN_API_KEY in an X-API-Key header.
Run it locally
Needs Rust >= 1.88 (rustup; on Linux also pkg-config and
libssl-dev) and PostgreSQL. Python 3.11 with uv
only for training models.
git clone https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor.git
cd Northern-Thailand-Ping-River-Monitor
docker run -d --name ping-pg -e POSTGRES_PASSWORD=ping -e POSTGRES_DB=ping -p 55432:5432 postgres:16
cp .env.example .env
# POSTGRES_CONNECTION_STRING=postgresql://postgres:ping@localhost:55432/ping
# WEB_LISTEN=127.0.0.1:8000
cargo build --release --manifest-path rust/Cargo.toml
rust/target/release/ping-collector check-db # create the tables
rust/target/release/ping-collector fill-gaps 30 # the last 30 days from RID
rust/target/release/ping-collector run # collect, forecast, serve http://127.0.0.1:8000
Notifications stay off until NTFY_SERVER or MATRIX_* is set. Without trained
models the forecast is the persistence heuristic, marked source: "heuristic".
Training needs years of history (the server's database, or FLOOD_ML_DB_URL) and
the Open-Meteo archive:
uv sync --python 3.11
uv run python scripts/train_flood_model.py --stations all # writes models/flood_*
uv run python scripts/evaluate_variants.py --stations P.1 # the rolling-origin harness
make help lists the rest (make check runs every CI gate locally).
Deployment
Production is one systemd unit on a small VPS behind Cloudflare; the app directory
/opt/thailand-water-monitor is a git checkout. To update:
cd /opt/thailand-water-monitor && git pull
cargo build --release --locked --manifest-path rust/Cargo.toml
sudo bash scripts/install.sh
scripts/install.sh (idempotent, also for a fresh host) installs the binary and
ping-collector.service, syncs the Python environment for the monthly
water-monitor-retrain.timer, runs a dry run of every source and a forecast check
against the Python reference, restarts the service and waits for the website. The
retrain trains into models/.staging, refuses to promote anything that is not a
rain-enabled (hgb-v3+) set covering the expected stations, and renames the
bundles into place; the service picks them up on its next forecast. Details and
the operations runbook: docs/FLOOD_FORECASTING.md
sections 6–8. Configuration: .env.example.
Repository layout
rust/collector/ ping-collector: collection, forecast run, ntfy/Matrix, website + API (src/web)
rust/forecast/ forecast inference from the exported models; parity tests against Python
src/static/ the dashboard (single file, EN/TH) and its map data
src/data/ the bundled station list
src/ml/ features, training, evaluation harness, export; predict.py is the reference
src/*.py HII and RID-dam clients/stores for the archive backfills
scripts/ install.sh, retrain.sh, train/evaluate/backfill tools, systemd units
tests/ pytest (synthetic data; database tests need TEST_DATABASE_URL)
docs/ model write-up, data sources, notifications, Matrix, rewrite history
models/ trained bundles (gitignored) and evaluation results
Documentation
- docs/FLOOD_FORECASTING.md — the model: data, features, evaluation, measured performance, negatives, deployment, retraining
- docs/DATA_SOURCES.md — every ingested and candidate source, endpoints, quirks
- docs/NOTIFICATIONS.md — public push alerts: topics, semantics, ntfy deployment
- docs/MATRIX_QUICK_START.md — the same alerts in a Matrix room
- rust/README.md — the service: commands, configuration, develop, deploy
- docs/RUST_REWRITE.md — why and how it moved from Python to Rust
- docs/DASHBOARD_REDESIGN.md — the dashboard: layout, rules, how it replaced the old page
- docs/references/NOTABLE_DOCUMENTS.md — official Thai government resources
- Public overview: buildfor.life/docs/tooling/ping-river-monitor
Contributing
make format before committing, make check must stay green; see
CONTRIBUTING.md. Issues and merge requests on
git.b4l.co.th.
Data sources and thanks
Royal Irrigation Department (RID) gauge telemetry and reservoir reports; Hydro-Informatics Institute (HII) / ThaiWater open API; Open-Meteo; OpenStreetMap contributors for the river geometry; the CMU Water Center (Chiang Mai University) for the flood hazard map of the 2024 flood that the P.1 stages and the city's flood zones come from. All instruments are theirs; we aggregate, store, fill gaps and forecast.
License
MIT — see LICENSE.