grabowski
04ebdaa931
CI / Format & lint (push) Successful in 29s
CI / Test suite (push) Successful in 50s
Docs / Validate documentation (push) Successful in 23s
Security / Dependency vulnerabilities (push) Successful in 1m34s
Security / Static analysis (push) Successful in 17s
Security / License report (push) Successful in 1m27s
RID's hourly report often lags 1-2 h in the evening. The cycle discarded the whole report unless it already held the current hour, the web-API loop then slept a full hour, and after 01:00 only today's report was fetched. Result: the 19:00-24:00 readings were dropped most nights (P.1 lost 00:00 on 23 of the last 60 days, 13 evening hours this week), the feed went stale from ~21:00 until morning and ntfy ping-status sent a stale/recovered pair daily. - run_scraping_cycle() upserts every reading the report has; "current hour present" now only decides the retry cadence (returned as before). - Yesterday's report is re-fetched while the store has holes in it (hours 22-24 are only reachable through that day's report). - Leader loop polls at :05 (RID_POLL_OFFSET_MINUTES) and retries every RID_RETRY_MINUTES (10) while RID is behind; HII/dams/rain once per clock hour; forecast precompute + ntfy when newer readings arrive or hourly. - Startup re-fetches holes from the last GAP_FILL_DAYS_ON_START (14) days. - /scraping/status: the leader shares its stats via a JSON file, so any worker reports them (was always 0 runs); adds feed_current, latest_reading, readings_saved.
165 lines
9.5 KiB
Markdown
165 lines
9.5 KiB
Markdown
# Northern Thailand Ping River Monitor
|
||
|
||
Live water levels, discharge, rainfall and machine-learning flood forecasts for the
|
||
Ping River basin around Chiang Mai. Collects hourly gauge data from public sources,
|
||
keeps the full history in PostgreSQL, and serves a bilingual dashboard, an open REST
|
||
API, and 6/12/24-hour flood-risk forecasts per gauge.
|
||
|
||
**Live: [water.buildfor.life](https://water.buildfor.life/)** · API reference at
|
||
[/docs](https://water.buildfor.life/docs) · built by [buildfor.life](https://buildfor.life)
|
||
after the [October 2024 flood](https://buildfor.life/blog/chiang-mai-flood-2024/) —
|
||
background in [Teaching a Model to See the Ping River Rise 13 Hours Early](https://buildfor.life/blog/ping-river-monitor/).
|
||
|
||
[](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
|
||
[](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
|
||
[](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
|
||
[](https://python.org)
|
||
[](LICENSE)
|
||
|
||
## What it does
|
||
|
||
- **Collects** hourly water level and discharge from 16 Royal Irrigation Department
|
||
(RID) telemetry gauges, Chiang Dao to the southern basin, since 2018-08; hourly
|
||
rainfall and water level from 400+ ThaiWater/HII stations; Open-Meteo catchment
|
||
rainfall (archive + 48 h forecast); daily Mae Ngat reservoir state. Every source and
|
||
its quirks: [docs/DATA_SOURCES.md](docs/DATA_SOURCES.md).
|
||
- **Fills gaps.** The raw RID grid had readings for ~56 % of hours; a full-history
|
||
re-fetch plus HII cross-fill brought it to ~93 %. `GET /api/stats` reports the
|
||
current figure.
|
||
- **Forecasts.** Per gauge and horizon, a gradient-boosted model predicts the rise
|
||
within 6/12/24 h and the probability of crossing the station's warning and danger
|
||
levels. Trained on the monitor's own history plus catchment rain; evaluated
|
||
rolling-origin, event by event. On the October 2024 record flood, trained only on
|
||
data through August 2024, the first alert came **13 hours before** P.1 crossed
|
||
3.70 m. Everything about the model, including what did not work:
|
||
[docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md).
|
||
- **Shows it.** A Leaflet map with the river drawn as OSM geometry and styled by live
|
||
discharge, rain gauges, the Chiang Mai inundation zones, per-station history, the
|
||
forecast card, a replay of the 2024 flood, English/Thai, light/dark.
|
||
- **Notifies.** Public push alerts over a self-hosted [ntfy](https://ntfy.sh): one
|
||
message when a gauge crosses its warning or danger level, one all-clear on the
|
||
way down, an opt-in early-warning topic from the model, nothing in between.
|
||
Subscribe from the free app, no account. Matrix room alerts for a team are
|
||
also supported.
|
||
|
||
## Quick start
|
||
|
||
Python **3.11** (3.13 breaks the pinned `psycopg2-binary`), PostgreSQL for anything
|
||
beyond a quick look, [uv](https://docs.astral.sh/uv/).
|
||
|
||
```bash
|
||
git clone https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor.git
|
||
cd Northern-Thailand-Ping-River-Monitor
|
||
uv sync --python 3.11
|
||
cp .env.example .env # DB_TYPE, POSTGRES_CONNECTION_STRING, optional MATRIX_*
|
||
uv run python run.py --web-api # dashboard + API on http://localhost:8000
|
||
```
|
||
|
||
`DB_TYPE=sqlite` works for the dashboard and API; the forecasting path expects the
|
||
PostgreSQL history.
|
||
|
||
```bash
|
||
uv run python run.py --status # collector status
|
||
uv run python run.py --test # one collection cycle
|
||
uv run python run.py --fill-gaps 7 # re-fetch the last 7 days from RID
|
||
uv run python run.py --collect-hii # one ThaiWater/HII collection cycle
|
||
uv run python run.py --alert-check # evaluate thresholds, notify Matrix
|
||
uv run python scripts/train_flood_model.py --stations all # retrain (~12 min)
|
||
make test # pytest, synthetic data, no network
|
||
make format # black + isort (the CI contract)
|
||
```
|
||
|
||
RID collection in `--web-api` mode: polled at :05 past each hour
|
||
(`RID_POLL_OFFSET_MINUTES`); while RID has not published the current hour yet,
|
||
retried every `RID_RETRY_MINUTES` (10). Every poll stores all readings the report
|
||
has, and yesterday's report is re-fetched while it still has holes (RID's evening
|
||
hours often arrive 1-2 h late). On startup the leader re-fetches holes from the
|
||
last `GAP_FILL_DAYS_ON_START` (14) days; for older history run `--fill-gaps all`.
|
||
|
||
## API
|
||
|
||
Read-only, no key, JSON. Base URL `https://water.buildfor.life`; timestamps are
|
||
Asia/Bangkok wall-clock without an offset suffix.
|
||
|
||
| Endpoint | Returns |
|
||
| --- | --- |
|
||
| `GET /stations` | The 16 RID gauges: code, Thai/English names, coordinates |
|
||
| `GET /measurements/latest?limit=N` | Newest reading per station |
|
||
| `GET /measurements/history/{code}?hours=N` | Hourly history; or `?start=YYYY-MM-DD&end=YYYY-MM-DD`; `limit` ≤ 100000 |
|
||
| `GET /forecast` | Current flood-risk forecast, every station × horizon, with thresholds and P.1 inundation-stage probabilities |
|
||
| `GET /api/forecast/history/{code}?hours=N&horizon=24` | Forecasts as issued, for auditing lead time after the fact |
|
||
| `GET /api/hii/rainfall/latest`, `/api/hii/waterlevel/latest` | Latest ThaiWater/HII gauge readings |
|
||
| `GET /api/hii/rainfall/catchment?days=N` | HII gauge catchment-mean rain next to the Open-Meteo series the model uses |
|
||
| `GET /api/forecast/skill?station_code=P.1` | Issued forecasts vs what happened, per deployed model version |
|
||
| `GET /api/notifications` | ntfy server and topic names for the subscribe panel |
|
||
| `GET /api/stats` | Row counts per source, date range, coverage |
|
||
| `GET /health` | DB / upstream / memory checks |
|
||
|
||
Interactive reference with schemas: [water.buildfor.life/docs](https://water.buildfor.life/docs).
|
||
Responses are cached briefly server-side; poll no faster than once a minute — the data
|
||
changes hourly.
|
||
|
||
## Deployment
|
||
|
||
Production is a systemd unit on a small VPS behind Cloudflare, updated by `git pull`.
|
||
`scripts/install.sh` (run as root from a checkout) creates the `water-monitor` user,
|
||
deploys to `/opt/thailand-water-monitor`, runs `uv sync` into `.venv`, installs
|
||
`water-monitor.service` and the monthly `water-monitor-retrain.timer`.
|
||
|
||
```bash
|
||
sudo bash scripts/install.sh
|
||
sudo systemctl start water-monitor.service
|
||
systemctl list-timers water-monitor-retrain.timer
|
||
```
|
||
|
||
The retrain timer runs `scripts/retrain.sh`, which trains into `models/.staging`,
|
||
refuses to promote anything that is not a rain-enabled (`hgb-v3+`) set covering the
|
||
expected stations, and renames the bundles into place. Details and the operations
|
||
runbook: [docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md) sections 6–8.
|
||
|
||
## Repository layout
|
||
|
||
```
|
||
src/ collector, API (web_api.py), dashboard (static/dashboard.html)
|
||
src/ml/ features, training, evaluation harness, prediction, rain/dam/HII loaders
|
||
scripts/ train_flood_model.py, retrain.sh, evaluate_variants.py, install.sh, dev_proxy.py
|
||
tests/ pytest suite (synthetic data; no DB or network)
|
||
docs/ FLOOD_FORECASTING.md, DATA_SOURCES.md, deployment and station guides
|
||
models/ trained bundles + metrics.json (gitignored) and evaluation results (tracked)
|
||
.gitea/workflows/ ci (format/lint/tests), security (pip-audit/bandit), docs (link + OpenAPI checks)
|
||
```
|
||
|
||
## Documentation
|
||
|
||
- [docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md) — the model: data, features, evaluation, measured performance, negatives, deployment, retraining
|
||
- [docs/DATA_SOURCES.md](docs/DATA_SOURCES.md) — every ingested and candidate source, endpoints, quirks
|
||
- [docs/STATION_MANAGEMENT_GUIDE.md](docs/STATION_MANAGEMENT_GUIDE.md) — adding/editing gauges
|
||
- [docs/DATABASE_DEPLOYMENT_GUIDE.md](docs/DATABASE_DEPLOYMENT_GUIDE.md), [POSTGRESQL_SETUP.md](POSTGRESQL_SETUP.md) — database setup
|
||
- [docs/NOTIFICATIONS.md](docs/NOTIFICATIONS.md) — public push alerts: topics, semantics, ntfy deployment
|
||
- [docs/MATRIX_QUICK_START.md](docs/MATRIX_QUICK_START.md) — Matrix room alerts for a team
|
||
- [docs/GAP_FILLING_GUIDE.md](docs/GAP_FILLING_GUIDE.md) — data integrity tooling
|
||
- [docs/references/NOTABLE_DOCUMENTS.md](docs/references/NOTABLE_DOCUMENTS.md) — official Thai government resources
|
||
- Public overview: [buildfor.life/docs/tooling/ping-river-monitor](https://buildfor.life/docs/tooling/ping-river-monitor/)
|
||
|
||
Other database backends (VictoriaMetrics, InfluxDB, MySQL, SQLite) and the Grafana
|
||
dashboards under `grafana/` are supported by the adapters but not what production
|
||
runs; see [docs/VICTORIAMETRICS_SETUP.md](docs/VICTORIAMETRICS_SETUP.md) if you want them.
|
||
|
||
## Contributing
|
||
|
||
`make format` before committing (black 88 columns, isort black profile — the CI gate),
|
||
`make test` must stay green, tests use synthetic data only. See
|
||
[CONTRIBUTING.md](CONTRIBUTING.md). Issues and merge requests on
|
||
[git.b4l.co.th](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor).
|
||
|
||
## Data sources and thanks
|
||
|
||
Royal Irrigation Department (RID) gauge telemetry; Hydro-Informatics Institute (HII) /
|
||
ThaiWater open API; Open-Meteo; OpenStreetMap contributors for the river geometry;
|
||
Chiang Mai Municipality for the inundation map the P.1 stages are keyed to. All
|
||
instruments are theirs; we aggregate, store, fill gaps and forecast.
|
||
|
||
## License
|
||
|
||
MIT — see [LICENSE](LICENSE).
|