# Rust: the service

Since October 2026 the whole Ping River Monitor service is one Rust binary,
`ping-collector`; Python is left with model training and evaluation
(`src/ml`) and the archive backfills. How it got here, and the parity rules
the rewrite followed: [docs/RUST_REWRITE.md](../docs/RUST_REWRITE.md).

| Crate | What |
| --- | --- |
| `collector` (`ping-collector`) | collection (RID, HII/ThaiWater, RID dams, Open-Meteo), the forecast run, ntfy + Matrix notifications, the website and API (`collector/src/web`) |
| `forecast` (`ping-forecast`, library) | flood-forecast inference from the models trained in Python: features, HII gap fill, gradient-boosted trees, heuristic fallback |

## ping-collector

```text
ping-collector run                  # the service (systemd): startup gap fill, RID at :05 (+10 min
                                    # retries while RID is behind), HII/dams/rain once per clock
                                    # hour; FORECAST_IN_COLLECTOR=true: forecast + ntfy/Matrix
                                    # after each cycle with newer readings; WEB_LISTEN: website/API
ping-collector serve [--listen A] [--no-analytics]
                                    # the website and API alone (default 127.0.0.1:8001)
ping-collector once [--dry-run]     # one cycle of every source; --dry-run writes nothing
                                    # and compares the RID fetch with the database
ping-collector fill-gaps <days|all> # re-fetch RID reports with holes
ping-collector check-db             # connect, create missing tables, row counts
ping-collector forecast [--json] [--save] [--now T] [--rain-file F]
                                    # forecast from the database and print it
ping-collector catch-up-forecasts [--hours N]
                                    # forecast the hours whose readings came late (RID's
                                    # overnight batch) as a run then would have; default 336 h
ping-collector notify --dry-run     # the readings the notifier judges, and what it would send
```

Configuration: the environment, or `.env` in the working directory
(`--env-file` for another), every key with its default in
[.env.example](../.env.example). `.env` is parsed with python-dotenv's rules
(`collector/src/envfile.rs`, verified on 614 generated inputs) because the
Python tools read the same file: an unparsable line is skipped with a warning
giving only its line number, never its value. Variables already set (e.g. by
systemd's `EnvironmentFile=`) win. All timestamps are Asia/Bangkok wall-clock,
whatever the server's time zone.

### Collection

The tables, upserts and parsers are the ones the Python collectors had (the
DDL is in `collector/src/db.rs`; the Python backfill stores create the same).
RID's evening report often lags 1-2 h: every poll stores every reading the
report has, yesterday's report is re-fetched while it has holes, and the
startup gap fill repairs the last `GAP_FILL_DAYS_ON_START` days.

### Forecast and notifications

After every cycle that brought newer RID or HII readings (and at least
hourly) the forecast runs over the last 336 h, is stored in `forecast_history`
(for skill tracking) and `forecast_current`, and the notifier compares the
readings and forecasts with `notification_state`.

- **Models** are trained in Python. `src/ml/export.py` writes
  `models/flood_<station>.json.gz` next to each joblib bundle (`train_all()`
  does it; `python -m src.ml.export` for existing bundles); the service
  reloads changed files on its next run, so a retrain needs no restart.
- **Shared constants** (thresholds, flood stages, upstream leads, notifier
  settings) are compiled in from `forecast/flood_config.json`, generated by
  `uv run python -m src.ml.export --config`. `tests/test_model_export.py`
  fails while it differs from Python; a change needs a rebuild.
- **ntfy** topics, texts and hysteresis: [docs/NOTIFICATIONS.md](../docs/NOTIFICATIONS.md).
  For the stations whose ThaiWater twin is the same sensor, a reading newer
  than RID's drives the level alerts, marked as ThaiWater telemetry. "Gauge
  feed stale" means no source has a reading for 3 h, except during RID's
  nightly pause (21:00 to `FEED_NIGHT_UNTIL_HOUR`, 9) while the river is
  quiet; RID alone lagging is reported once after `RID_STALE_ALERT_HOURS`.
- **Matrix** (`MATRIX_*`): the room gets the same transitions (per station,
  outlook, feed; not ntfy's per-topic summaries), each once, tracked under
  its own `matrix:` keys in `notification_state`, so a Matrix outage is
  retried on Matrix only. [docs/MATRIX_QUICK_START.md](../docs/MATRIX_QUICK_START.md).
- Memory: under 100 MB resident in production, all 15 models loaded
  (24-byte tree nodes); the unit allows 512 MB.

### Website and API

`collector/src/web` (axum) serves the dashboard (`STATIC_DIR`/dashboard.html,
default `src/static`, loaded at start), `/static`, the crawler files,
`/docs` + `/redoc` + `/openapi.json`, and the read API, with gzip,
Cache-Control per path, CORS (`CORS_ALLOW_ORIGINS`), Umami API tracking
(`UMAMI_*`) and stale-while-revalidate response caches with byte budgets.
Station editing, `/config` and `POST /scrape/trigger` need `ADMIN_API_KEY`
(`X-API-Key` header).

- Parameters are declared once per route (`web/params.rs`): the same list
  validates the request (pydantic-style 422s) and builds the OpenAPI
  document (`web/openapi.rs`), so `/docs` cannot drift from what is accepted.
- The response formats are the retired FastAPI app's, kept so clients did
  not notice the switch: field order, `isoformat()` timestamps, Python's
  `round()`, numpy's pairwise sums for the catchment means (`web/pyfmt.rs`),
  `{"detail": ...}` errors.
- An old stored forecast is served marked `X-Data-Stale: true`; `/health`
  counts current ThaiWater telemetry of the same gauges while RID lags.

## Develop

```bash
cd rust
cargo fmt --all && cargo clippy --all-targets -- -D warnings
docker run -d --name ping-pg -e POSTGRES_PASSWORD=ping -e POSTGRES_DB=ping -p 55432:5432 postgres:16
TEST_DATABASE_URL=postgres://postgres:ping@localhost:55432/ping cargo test
CARGO_TARGET_DIR=target/msrv cargo +1.88 check --all-targets --locked   # the minimum version
```

Without `TEST_DATABASE_URL` the database tests skip. CI
(`.gitea/workflows/rust.yml`) runs them against a PostgreSQL service and
uploads the Linux binary as the `ping-collector-linux-x86_64` artifact.

- The forecast's parity with the Python reference: `forecast/tests/parity.rs`
  replays fixtures from the Python production path
  (`forecast/testdata/gen_forecast_fixtures.py`; regenerate after a change to
  `src/ml/features.py`, `predict.py`, the HII fill or the dam frame). Features
  and levels must stay bit-identical, probabilities within a few ulp.
  `scripts/forecast_parity.py` compares the two on a live database.
- The notifier's texts are pinned by `collector/testdata/notify_python_cases.json.gz`
  (751 messages recorded from the Python notifier before it was retired).

HTTP uses native-tls (OpenSSL on Linux), not rustls: app.rid.go.th (dams)
only offers TLS 1.2 CBC cipher suites, which rustls refuses. Building on
Linux needs `pkg-config` and `libssl-dev`.

## Deploy

Needs Rust >= 1.88. Distro `cargo` packages are older (`resolver 3 is not
valid` / `edition 2024` errors), so install the toolchain once with rustup:

```bash
apt-get install -y build-essential pkg-config libssl-dev curl
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
. "$HOME/.cargo/env"        # puts ~/.cargo/bin before /usr/bin/cargo
cargo --version             # >= 1.88
```

Then on the server, in the checkout:

```bash
git pull
cargo build --release --locked --manifest-path rust/Cargo.toml   # or BIN=<CI artifact>
sudo bash scripts/install.sh
journalctl -u ping-collector -f
```

`scripts/install.sh` installs the binary and the units, syncs the Python
training environment, runs a dry run of every source and the forecast check,
restarts the service and waits for the website (details in its header).