docs: rewrite the README for the Rust service

README.md

@@ -2,8 +2,9 @@
 
 Live water levels, discharge, rainfall and machine-learning flood forecasts for the
 Ping River basin around Chiang Mai. Collects hourly gauge data from public sources,
-keeps the full history in PostgreSQL, and serves a bilingual dashboard, an open REST
+keeps the full history in PostgreSQL, serves a bilingual dashboard and an open REST
-API, and 6/12/24-hour flood-risk forecasts per gauge.
+API, forecasts the flood risk per gauge 6/12/24 hours ahead, and pushes alerts when
+a gauge crosses its warning or danger level.
 
 **Live: [water.buildfor.life](https://water.buildfor.life/)** · API reference at
 [/docs](https://water.buildfor.life/docs) · built by [buildfor.life](https://buildfor.life)
@@ -11,9 +12,9 @@ after the [October 2024 flood](https://buildfor.life/blog/chiang-mai-flood-2024/
 background in [Teaching a Model to See the Ping River Rise 13 Hours Early](https://buildfor.life/blog/ping-river-monitor/).
 
 [![CI](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/ci.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
+[![Rust](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/rust.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
 [![Security](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/security.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
 [![Docs](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions/workflows/docs.yml/badge.svg)](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor/actions)
-[![Python 3.11](https://img.shields.io/badge/Python-3.11-blue.svg)](https://python.org)
 [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
 
 ## What it does
@@ -21,11 +22,12 @@ background in [Teaching a Model to See the Ping River Rise 13 Hours Early](https
 - **Collects** hourly water level and discharge from 16 Royal Irrigation Department
   (RID) telemetry gauges, Chiang Dao to the southern basin, since 2018-08; hourly
   rainfall and water level from 400+ ThaiWater/HII stations; Open-Meteo catchment
-  rainfall (archive + 48 h forecast); daily Mae Ngat reservoir state. Every source and
+  rainfall (archive + 48 h forecast); daily state of the large RID dams, Mae Ngat
-  its quirks: [docs/DATA_SOURCES.md](docs/DATA_SOURCES.md).
+  above all. Every source and its quirks: [docs/DATA_SOURCES.md](docs/DATA_SOURCES.md).
 - **Fills gaps.** The raw RID grid had readings for ~56 % of hours; a full-history
-  re-fetch plus HII cross-fill brought it to ~93 %. `GET /api/stats` reports the
+  re-fetch plus HII cross-fill brought it to ~93 %. RID's report often arrives
-  current figure.
+  hours late in the evening, so late readings are always stored and holes are
+  re-fetched. `GET /api/stats` reports the current coverage.
 - **Forecasts.** Per gauge and horizon, a gradient-boosted model predicts the rise
   within 6/12/24 h and the probability of crossing the station's warning and danger
   levels. Trained on the monitor's own history plus catchment rain; evaluated
@@ -39,42 +41,38 @@ background in [Teaching a Model to See the Ping River Rise 13 Hours Early](https
 - **Notifies.** Public push alerts over a self-hosted [ntfy](https://ntfy.sh): one
   message when a gauge crosses its warning or danger level, one all-clear on the
   way down, an opt-in early-warning topic from the model, nothing in between.
-  Subscribe from the free app, no account. Matrix room alerts for a team are
+  Subscribe from the free app, no account. A Matrix room can get the same
-  also supported.
+  messages for a team. [docs/NOTIFICATIONS.md](docs/NOTIFICATIONS.md).
 
-## Quick start
+## How it runs
 
-Python **3.11** (3.13 breaks the pinned `psycopg2-binary`), PostgreSQL for anything
+```text
-beyond a quick look, [uv](https://docs.astral.sh/uv/).
+ RID gauges · ThaiWater/HII · RID dams · Open-Meteo
-
+                    │ hourly
-```bash
+                    ▼
-git clone https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor.git
+ ┌──────────────────────────────────────┐        ┌───────────────────────────┐
-cd Northern-Thailand-Ping-River-Monitor
+ │ ping-collector  (Rust, one process)  │        │ monthly retrain (Python)  │
-uv sync --python 3.11
+ │  collect → forecast → notify         │◄───────│  scikit-learn, exports    │
-cp .env.example .env            # DB_TYPE, POSTGRES_CONNECTION_STRING, optional MATRIX_*
+ │  website + API on :8000              │ models │  models/flood_*.json.gz   │
-uv run python run.py --web-api  # dashboard + API on http://localhost:8000
+ └──────┬──────────────┬────────────────┘        └─────────────┬─────────────┘
+        │              │                                       │
+        ▼              ▼                                       │
+  ntfy topics,    PostgreSQL ◄──────────── history ────────────┘
+  Matrix room     (gauges, rain, dams, forecasts, alert state)
 ```
 
-`DB_TYPE=sqlite` works for the dashboard and API; the forecasting path expects the
+- **The service** is one Rust binary, `ping-collector` ([rust/README.md](rust/README.md)),
-PostgreSQL history.
+  run by systemd. It polls RID at :05 past each hour and retries every 10 minutes
-
+  while the report is behind, fetches HII, the dams and the rain once an hour,
-```bash
+  forecasts after every cycle that brought new readings, sends the notifications,
-uv run python run.py --status          # collector status
+  and serves the dashboard and API, in under 100 MB of memory with every model
-uv run python run.py --test            # one collection cycle
+  loaded.
-uv run python run.py --fill-gaps 7     # re-fetch the last 7 days from RID
+- **The models** are trained in Python (`src/ml`, scikit-learn
-uv run python run.py --collect-hii     # one ThaiWater/HII collection cycle
+  HistGradientBoosting) by a monthly timer and exported for the Rust forecaster,
-uv run python run.py --alert-check     # evaluate thresholds, notify Matrix
+  which reproduces Python's features bit for bit and reloads new models without
-uv run python scripts/train_flood_model.py --stations all   # retrain (~12 min)
+  a restart.
-make test                              # pytest, synthetic data, no network
+- Until October 2026 all of this was a Python/FastAPI app; how and why it moved:
-make format                            # black + isort (the CI contract)
+  [docs/RUST_REWRITE.md](docs/RUST_REWRITE.md).
-```
-
-RID collection in `--web-api` mode: polled at :05 past each hour
-(`RID_POLL_OFFSET_MINUTES`); while RID has not published the current hour yet,
-retried every `RID_RETRY_MINUTES` (10). Every poll stores all readings the report
-has, and yesterday's report is re-fetched while it still has holes (RID's evening
-hours often arrive 1-2 h late). On startup the leader re-fetches holes from the
-last `GAP_FILL_DAYS_ON_START` (14) days; for older history run `--fill-gaps all`.
 
 ## API
 
@@ -88,76 +86,112 @@ Asia/Bangkok wall-clock without an offset suffix.
 | `GET /measurements/history/{code}?hours=N` | Hourly history; or `?start=YYYY-MM-DD&end=YYYY-MM-DD`; `limit` ≤ 100000 |
 | `GET /forecast` | Current flood-risk forecast, every station × horizon, with thresholds and P.1 inundation-stage probabilities |
 | `GET /api/forecast/history/{code}?hours=N&horizon=24` | Forecasts as issued, for auditing lead time after the fact |
+| `GET /api/forecast/skill?station_code=P.1` | Issued forecasts vs what happened, per deployed model version |
 | `GET /api/hii/rainfall/latest`, `/api/hii/waterlevel/latest` | Latest ThaiWater/HII gauge readings |
 | `GET /api/hii/rainfall/catchment?days=N` | HII gauge catchment-mean rain next to the Open-Meteo series the model uses |
-| `GET /api/forecast/skill?station_code=P.1` | Issued forecasts vs what happened, per deployed model version |
 | `GET /api/notifications` | ntfy server and topic names for the subscribe panel |
 | `GET /api/stats` | Row counts per source, date range, coverage |
-| `GET /health` | DB / upstream / memory checks |
+| `GET /health` | Database and data freshness (RID, ThaiWater as backup), the RID API, memory |
+| `GET /scraping/status` | Collection runs, the last forecast run, service version |
 
-Interactive reference with schemas: [water.buildfor.life/docs](https://water.buildfor.life/docs).
+Interactive reference with schemas: [water.buildfor.life/docs](https://water.buildfor.life/docs)
-Responses are cached briefly server-side; poll no faster than once a minute — the data
+(also `/redoc`, `/openapi.json`). Responses are cached briefly server-side; poll no
-changes hourly.
+faster than once a minute — the data changes hourly. Station editing
+(`POST/PUT/DELETE /stations`), `/config` and `POST /scrape/trigger` need the
+server's `ADMIN_API_KEY` in an `X-API-Key` header.
+
+## Run it locally
+
+Needs Rust >= 1.88 ([rustup](https://rustup.rs); on Linux also `pkg-config` and
+`libssl-dev`) and PostgreSQL. Python 3.11 with [uv](https://docs.astral.sh/uv/)
+only for training models.
+
+```bash
+git clone https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor.git
+cd Northern-Thailand-Ping-River-Monitor
+docker run -d --name ping-pg -e POSTGRES_PASSWORD=ping -e POSTGRES_DB=ping -p 55432:5432 postgres:16
+cp .env.example .env
+#   POSTGRES_CONNECTION_STRING=postgresql://postgres:ping@localhost:55432/ping
+#   WEB_LISTEN=127.0.0.1:8000
+cargo build --release --manifest-path rust/Cargo.toml
+rust/target/release/ping-collector check-db         # create the tables
+rust/target/release/ping-collector fill-gaps 30     # the last 30 days from RID
+rust/target/release/ping-collector run              # collect, forecast, serve http://127.0.0.1:8000
+```
+
+Notifications stay off until `NTFY_SERVER` or `MATRIX_*` is set. Without trained
+models the forecast is the persistence heuristic, marked `source: "heuristic"`.
+Training needs years of history (the server's database, or `FLOOD_ML_DB_URL`) and
+the Open-Meteo archive:
+
+```bash
+uv sync --python 3.11
+uv run python scripts/train_flood_model.py --stations all    # writes models/flood_*
+uv run python scripts/evaluate_variants.py --stations P.1     # the rolling-origin harness
+```
+
+`make help` lists the rest (`make check` runs every CI gate locally).
 
 ## Deployment
 
-Production is a systemd unit on a small VPS behind Cloudflare, updated by `git pull`.
+Production is one systemd unit on a small VPS behind Cloudflare; the app directory
-`scripts/install.sh` (run as root from a checkout) creates the `water-monitor` user,
+`/opt/thailand-water-monitor` is a git checkout. To update:
-deploys to `/opt/thailand-water-monitor`, runs `uv sync` into `.venv`, installs
-`water-monitor.service` and the monthly `water-monitor-retrain.timer`.
 
 ```bash
+cd /opt/thailand-water-monitor && git pull
+cargo build --release --locked --manifest-path rust/Cargo.toml
 sudo bash scripts/install.sh
-sudo systemctl start water-monitor.service
-systemctl list-timers water-monitor-retrain.timer
 ```
 
-The retrain timer runs `scripts/retrain.sh`, which trains into `models/.staging`,
+`scripts/install.sh` (idempotent, also for a fresh host) installs the binary and
-refuses to promote anything that is not a rain-enabled (`hgb-v3+`) set covering the
+`ping-collector.service`, syncs the Python environment for the monthly
-expected stations, and renames the bundles into place. Details and the operations
+`water-monitor-retrain.timer`, runs a dry run of every source and a forecast check
-runbook: [docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md) sections 6–8.
+against the Python reference, restarts the service and waits for the website. The
+retrain trains into `models/.staging`, refuses to promote anything that is not a
+rain-enabled (`hgb-v3+`) set covering the expected stations, and renames the
+bundles into place; the service picks them up on its next forecast. Details and
+the operations runbook: [docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md)
+sections 6–8. Configuration: [.env.example](.env.example).
 
 ## Repository layout
 
 ```
-src/                 collector, API (web_api.py), dashboard (static/dashboard.html)
+rust/collector/      ping-collector: collection, forecast run, ntfy/Matrix, website + API (src/web)
-src/ml/              features, training, evaluation harness, prediction, rain/dam/HII loaders
+rust/forecast/       forecast inference from the exported models; parity tests against Python
-scripts/             train_flood_model.py, retrain.sh, evaluate_variants.py, install.sh, dev_proxy.py
+src/static/          the dashboard (single file, EN/TH) and its map data
-tests/               pytest suite (synthetic data; no DB or network)
+src/data/            the bundled station list
-docs/                FLOOD_FORECASTING.md, DATA_SOURCES.md, deployment and station guides
+src/ml/              features, training, evaluation harness, export; predict.py is the reference
-models/              trained bundles + metrics.json (gitignored) and evaluation results (tracked)
+src/*.py             HII and RID-dam clients/stores for the archive backfills
-.gitea/workflows/    ci (format/lint/tests), security (pip-audit/bandit), docs (link + OpenAPI checks)
+scripts/             install.sh, retrain.sh, train/evaluate/backfill tools, systemd units
+tests/               pytest (synthetic data; database tests need TEST_DATABASE_URL)
+docs/                model write-up, data sources, notifications, Matrix, rewrite history
+models/              trained bundles (gitignored) and evaluation results
 ```
 
 ## Documentation
 
 - [docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md) — the model: data, features, evaluation, measured performance, negatives, deployment, retraining
 - [docs/DATA_SOURCES.md](docs/DATA_SOURCES.md) — every ingested and candidate source, endpoints, quirks
-- [docs/STATION_MANAGEMENT_GUIDE.md](docs/STATION_MANAGEMENT_GUIDE.md) — adding/editing gauges
-- [docs/DATABASE_DEPLOYMENT_GUIDE.md](docs/DATABASE_DEPLOYMENT_GUIDE.md), [POSTGRESQL_SETUP.md](POSTGRESQL_SETUP.md) — database setup
 - [docs/NOTIFICATIONS.md](docs/NOTIFICATIONS.md) — public push alerts: topics, semantics, ntfy deployment
-- [docs/MATRIX_QUICK_START.md](docs/MATRIX_QUICK_START.md) — Matrix room alerts for a team
+- [docs/MATRIX_QUICK_START.md](docs/MATRIX_QUICK_START.md) — the same alerts in a Matrix room
-- [docs/GAP_FILLING_GUIDE.md](docs/GAP_FILLING_GUIDE.md) — data integrity tooling
+- [rust/README.md](rust/README.md) — the service: commands, configuration, develop, deploy
+- [docs/RUST_REWRITE.md](docs/RUST_REWRITE.md) — why and how it moved from Python to Rust
 - [docs/references/NOTABLE_DOCUMENTS.md](docs/references/NOTABLE_DOCUMENTS.md) — official Thai government resources
 - Public overview: [buildfor.life/docs/tooling/ping-river-monitor](https://buildfor.life/docs/tooling/ping-river-monitor/)
 
-Other database backends (VictoriaMetrics, InfluxDB, MySQL, SQLite) and the Grafana
-dashboards under `grafana/` are supported by the adapters but not what production
-runs; see [docs/VICTORIAMETRICS_SETUP.md](docs/VICTORIAMETRICS_SETUP.md) if you want them.
-
 ## Contributing
 
-`make format` before committing (black 88 columns, isort black profile — the CI gate),
+`make format` before committing, `make check` must stay green; see
-`make test` must stay green, tests use synthetic data only. See
 [CONTRIBUTING.md](CONTRIBUTING.md). Issues and merge requests on
 [git.b4l.co.th](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor).
 
 ## Data sources and thanks
 
-Royal Irrigation Department (RID) gauge telemetry; Hydro-Informatics Institute (HII) /
+Royal Irrigation Department (RID) gauge telemetry and reservoir reports;
-ThaiWater open API; Open-Meteo; OpenStreetMap contributors for the river geometry;
+Hydro-Informatics Institute (HII) / ThaiWater open API; Open-Meteo; OpenStreetMap
-Chiang Mai Municipality for the inundation map the P.1 stages are keyed to. All
+contributors for the river geometry; Chiang Mai Municipality for the inundation map
-instruments are theirs; we aggregate, store, fill gaps and forecast.
+the P.1 stages are keyed to. All instruments are theirs; we aggregate, store, fill
+gaps and forecast.
 
 ## License