Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/)
collects, forecasts, notifies (ntfy + Matrix) and serves the website/API;
nothing in production ran the Python service code any more.
Removed (~17k lines): the FastAPI app (web_api, schemas, health_check,
metrics, postgres_history, forecast_history, thaiwater), the Python RID
collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix
alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL,
InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/
logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling
(api_parity, collector_parity, e2e_notify, install_collector.sh,
gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files,
sql/, the release workflow and the docs for all of that. Their tests went
with them.
Kept in Python: src/ml (training, evaluation harness, export; predict.py as
the reference the Rust forecaster is tested against), the HII and RID-dam
clients/stores for the archive backfills (their live collectors removed),
the backfill scripts, retrain.sh + timer, forecast_parity.py.
- PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env;
the stores lost their sqlite/mysql branches (and the 2026-08 column
migration). The database tests run against a throwaway schema from
TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the
Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB).
- The notifier settings moved from src/notify.py into src/ml/export.py;
flood_config.json regenerates byte-identical.
- tests/test_dashboard.py checks every path the dashboard fetches against
ping-collector's router.
- Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil,
schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv
1.1.1 explicit (it came in through uvicorn[standard]).
- Formatting contract extended to scripts/ and tests/ (reformatted once).
- scripts/install.sh is the one installer (fresh host or update, in place in
a checkout without touching its ownership): binary, units, uv sync, .env
defaults, removes the retired water-monitor.service and any old
--alert-check cron job, dry run + forecast check, restart, readiness wait.
- Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments
and Rust doc comments updated to the Rust service.
train_all() now raises RainUnavailableError when use_rain=True and the
Open-Meteo history cannot be loaded, instead of logging a warning and
writing gauge-only (v2) bundles over the deployed v3 set -- which is what
the 2026-09-01 server retrain did unnoticed. --no-rain remains the explicit
way to get v2. CLI exits 2 with a one-line error. Three tests cover the
guard, the opt-out, and the v3 happy path.
scripts/retrain.sh trains into models/.staging, refuses to promote unless
metrics.json shows hgb-v3+ and >=14 trained stations, then renames bundles
into place (previous generation kept in models/.previous). No API restart:
predict.py reloads by mtime on the hourly precompute.
water-monitor-retrain.{service,timer}: 1st of each month 03:30, Persistent,
OMP_NUM_THREADS=4, Nice=15, same sandbox as the API unit. install.sh now
does `uv sync` into .venv (one env rule; removes a stale venv/) and enables
the timer. water-monitor.service in the repo matched neither the deployed
unit nor the uv env; it now does (run.py --web-api, .venv, EnvironmentFile).
- .env now chmod 0600 and APP_DIR chmod 0750 after chown, so the Matrix token
and DB credentials are not world-readable.
- uv auto-install (curl | sh as root) is now opt-in via AUTO_INSTALL_UV=1 and
pins a specific uv version; otherwise the script requires uv to be
pre-installed and fails with instructions, avoiding unattended remote code
execution as root.
- scripts/install.sh: one-command hardened deploy (creates the water-monitor
system user, deploys to /opt, builds a uv-managed venv, installs and enables
the systemd unit). Idempotent; excludes .env/*.db/stations.json from sync so
runtime state is preserved.
- Fix placeholder Documentation= URL in water-monitor.service.
- README: document the script as the primary systemd install path, with manual
steps kept as a fallback.
grabowski