# Rust services Rewrite of the Ping River Monitor in Rust, one service at a time. The plan, costs and the parity rules are in [docs/RUST_REWRITE.md](../docs/RUST_REWRITE.md). | Crate | Replaces | Status | | --- | --- | --- | | `collector` (`ping-collector`) | RID, HII/ThaiWater, RID dams and Open-Meteo collection in the Python web app | phase 1: in production since 2026-10-17 | | `forecast` (`ping-forecast`, library) | the flood forecast (`src/ml` serving path); run by `ping-collector` with ntfy | phase 2: in production since 2026-10-18 | | `collector`: `src/web` | the FastAPI app (`src/web_api.py`): dashboard, read API, `/docs` | phase 3: built, cutover pending | ## ping-collector Collects into the same PostgreSQL tables as the Python app, so the two can be swapped (and swapped back) with no data migration. The Python web app keeps serving the API, forecasting and sending ntfy alerts with `COLLECTION_ENABLED=false`, and re-forecasts whenever the newest RID reading in the database advances. ```text ping-collector run # the service: startup gap fill, RID at :05 (+10 min retries # while RID is behind), HII/dams/rain once per clock hour; # with FORECAST_IN_COLLECTOR=true also forecast + ntfy/Matrix; # with WEB_LISTEN the website and API ping-collector serve [--listen A] [--no-analytics] # the website and API only (default 127.0.0.1:8001) ping-collector once [--dry-run] # one cycle of every source; --dry-run writes nothing # and compares the RID fetch with the database ping-collector fill-gaps # re-fetch RID reports with holes ping-collector check-db # connect, create missing tables, row counts ping-collector forecast [--json] [--save] [--now T] [--rain-file F] # forecast from the database and print it ping-collector notify --dry-run # the readings the notifier judges, and what it would send ``` Configuration is the Python app's `.env` (read from the working directory, or `--env-file`) with python-dotenv's rules, verified on 614 generated inputs (`collector/testdata/`): an unparsable line is skipped with a warning giving only its line number, never its value. Variables already set (e.g. by systemd's `EnvironmentFile=`) win. Used keys: `POSTGRES_CONNECTION_STRING` (or `POSTGRES_HOST/PORT/DB/USER/PASSWORD`), `RID_POLL_OFFSET_MINUTES` (5), `RID_RETRY_MINUTES` (10), `GAP_FILL_DAYS_ON_START` (14), `ENABLE_HII_COLLECTION`, `ENABLE_RESERVOIR_COLLECTION`, `ENABLE_RAIN_COLLECTION`, `HII_BASIN_CODE` (6), `STATION_CONFIG_PATH` (falls back to the bundled `src/data/stations.json`), `SCRAPING_STATUS_FILE` (status JSON for `/scraping/status` when the website runs elsewhere), `LOG_LEVEL` / `RUST_LOG`; for phase 2 `FORECAST_IN_COLLECTOR` (false), `MODELS_DIR` (`models`), `NTFY_SERVER` / `NTFY_PUBLISH_URL` / `NTFY_TOPIC_PREFIX` / `NTFY_TOKEN` / `PUBLIC_URL` (as the Python app), `RID_STALE_ALERT_HOURS` (18, 0 = off). All timestamps are Asia/Bangkok wall-clock, whatever the server's time zone. Not ported: the one-off archive backfills (`scripts/backfill_*.py`), which stay in Python. ### Verified - 20 tests: parsers, gap/schedule logic, and database tests against a real PostgreSQL (late reports, midnight recovery, upsert revisions, NUMERIC rounding, dam metadata preservation). - Live parity: the Python collectors and `ping-collector` run back-to-back against the real upstream APIs into two empty databases produced identical rows in all 9 tables (16 stations, 1,104 RID readings, 103 + 76 HII rows, 35 dams x 2 days, 168 rain hours). Rerun with `scripts/collector_parity.py` after changing a parser, table or upsert on either side. - The Python API with `COLLECTION_ENABLED=false` served `/health`, `/measurements/latest`, `/api/stats`, `/scraping/status` and model forecasts from a database filled only by `ping-collector`. ## Forecast and notifications (phase 2) With `FORECAST_IN_COLLECTOR=true` in the shared `.env`, `ping-collector` runs the flood forecast after every cycle that brought newer RID or HII readings (and at least hourly), stores it in `forecast_history` and `forecast_current` (the Python API serves the latter as `/forecast`, falling back to computing itself when it is older than `FORECAST_MAX_AGE_SECONDS`), and sends the ntfy notifications. The Python app then neither forecasts nor notifies. - **Models** stay trained in Python. `src/ml/export.py` writes `models/flood_.json.gz` next to each joblib bundle (`train_all()` does it, `retrain.sh` promotes it, `python -m src.ml.export` exports existing bundles); the collector reloads changed files on its next run. - **Shared constants** (thresholds, flood stages, upstream leads, notify settings) are compiled in from `forecast/flood_config.json`, generated by `uv run python -m src.ml.export --config`. `tests/test_model_export.py` fails while it differs from Python, and a change needs a collector rebuild. - **Notifications**: same topics, texts, hysteresis and `notification_state` keys as `src/notify.py`, so switching sends nothing twice. One change: for the stations whose ThaiWater twin is the same sensor, a reading newer than RID's (RID's report lags 2-10 h at night) drives the level alerts, marked "ThaiWater telemetry; the RID report is not in yet". "Gauge feed stale" now means no source has a reading for 3 h; RID alone lagging is reported once after `RID_STALE_ALERT_HOURS`. - Memory: 70 MB peak with all 15 models loaded (24-byte tree nodes). Verified: - `forecast/tests/parity.rs` replays fixtures from the Python production path (`forecast/testdata/gen_forecast_fixtures.py`: the 2024 flood rising and at its peak with RID lagging behind HII, a quiet window with dam data, a stale-feed window, moved thresholds). HII fill, every feature and every level are bit-identical; with all 15 local models 377 of 384 probabilities are bit-identical and the rest within 2 ulp (`exp`). CI checks the P.1 model. - `scripts/forecast_parity.py` does the same on the live database: Python and `ping-collector forecast --json` at the same moment with the same rain series. The install script runs it before `--cutover-forecast`. - `collector/testdata/gen_notify_cases.py`: 652 hourly steps through `src/notify.py` (thresholds, capacity guard, outlook, feed gaps, ntfy outages); the Rust notifier sends the same 751 messages word for word. ## Website and API (phase 3) `src/web` serves what the FastAPI app served: the dashboard (`STATIC_DIR`/dashboard.html, default `src/static`, loaded at start), the crawler files, `/static`, `/docs` + `/redoc` + `/openapi.json`, and every API route with the same parameters, JSON (field order included), status codes, `{"detail": ...}` errors, Cache-Control, gzip, CORS (`CORS_ALLOW_ORIGINS`), Umami API tracking (`UMAMI_*`) and response caches (`*_CACHE_TTL_SECONDS`). Admin routes need `ADMIN_API_KEY`. With `WEB_LISTEN=0.0.0.0:8000` it runs inside `ping-collector run` (one process collects, forecasts, notifies and serves; `/scraping/status` and `POST /scrape/trigger` then talk to the collection loop directly); `serve` runs it alone. - Parameters are declared once per route (`web/params.rs`): the same list validates the request (pydantic-style 422s) and builds the OpenAPI document (`web/openapi.rs`), so `/docs` cannot drift from what is accepted. The component schemas are FastAPI's (`web/openapi_components.json`). - Python's output conventions are kept where they show: `isoformat()` timestamps, `round()`, numpy's pairwise sums for the catchment means (`web/pyfmt.rs`), the skill metrics' summation order. - `scripts/api_parity.py --python URL --rust URL` compares the two live on one database; `install_collector.sh --check-api` / `--cutover-api` run it on the server against `ping-collector serve` on port 8001. Matrix (`MATRIX_HOMESERVER`, `MATRIX_ACCESS_TOKEN`, `MATRIX_ROOM_ID`): the notifier sends the room the same transitions as ntfy (per station, outlook, feed; not ntfy's per-topic summaries), each once, with its own `matrix:` keys in `notification_state` (seeded from the ntfy ones on first use, so nothing already announced is repeated). It replaces the `run.py --alert-check` cron job, which the install script comments out. ### Develop ```bash cd rust cargo fmt --all && cargo clippy --all-targets -- -D warnings docker run -d --name ping-pg -e POSTGRES_PASSWORD=ping -e POSTGRES_DB=ping -p 55432:5432 postgres:16 TEST_DATABASE_URL=postgres://postgres:ping@localhost:55432/ping cargo test ``` Without `TEST_DATABASE_URL` the database tests skip. CI (`.gitea/workflows/rust.yml`) runs them against a PostgreSQL service and uploads the Linux binary as the `ping-collector-linux-x86_64` artifact. HTTP uses native-tls (OpenSSL on Linux), not rustls: app.rid.go.th (dams) only offers TLS 1.2 CBC cipher suites, which rustls refuses. Building on Linux needs `pkg-config` and `libssl-dev`. ### Deploy Needs Rust >= 1.88. Distro `cargo` packages are older (`resolver 3 is not valid` / `edition 2024` errors), so install the toolchain once with rustup: ```bash apt-get install -y build-essential pkg-config libssl-dev curl curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal . "$HOME/.cargo/env" # puts ~/.cargo/bin before /usr/bin/cargo cargo --version # >= 1.88 ``` Then on the server, after `git pull` (the Python side of the switch ships in the same commit): ```bash cargo build --release --locked --manifest-path rust/Cargo.toml # or BIN= sudo bash scripts/install_collector.sh # install + checks, nothing switched sudo bash scripts/install_collector.sh --cutover # Rust collects from now on sudo bash scripts/install_collector.sh --cutover-forecast # Rust also forecasts and notifies sudo bash scripts/install_collector.sh --check-api # compare the Rust API with FastAPI sudo bash scripts/install_collector.sh --cutover-api # Rust serves the website on :8000 sudo bash scripts/install_collector.sh --rollback-api # FastAPI serves the website again sudo bash scripts/install_collector.sh --rollback-forecast # forecasts + ntfy back to Python sudo bash scripts/install_collector.sh --rollback # Python does everything again journalctl -u ping-collector -f ``` Every install exports the models and runs `scripts/forecast_parity.py`; `--cutover-forecast` refuses to switch if it fails (`FORCE=1` overrides).