Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/)
collects, forecasts, notifies (ntfy + Matrix) and serves the website/API;
nothing in production ran the Python service code any more.
Removed (~17k lines): the FastAPI app (web_api, schemas, health_check,
metrics, postgres_history, forecast_history, thaiwater), the Python RID
collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix
alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL,
InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/
logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling
(api_parity, collector_parity, e2e_notify, install_collector.sh,
gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files,
sql/, the release workflow and the docs for all of that. Their tests went
with them.
Kept in Python: src/ml (training, evaluation harness, export; predict.py as
the reference the Rust forecaster is tested against), the HII and RID-dam
clients/stores for the archive backfills (their live collectors removed),
the backfill scripts, retrain.sh + timer, forecast_parity.py.
- PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env;
the stores lost their sqlite/mysql branches (and the 2026-08 column
migration). The database tests run against a throwaway schema from
TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the
Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB).
- The notifier settings moved from src/notify.py into src/ml/export.py;
flood_config.json regenerates byte-identical.
- tests/test_dashboard.py checks every path the dashboard fetches against
ping-collector's router.
- Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil,
schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv
1.1.1 explicit (it came in through uvicorn[standard]).
- Formatting contract extended to scripts/ and tests/ (reformatted once).
- scripts/install.sh is the one installer (fresh host or update, in place in
a checkout without touching its ownership): binary, units, uv sync, .env
defaults, removes the retired water-monitor.service and any old
--alert-check cron job, dry run + forecast check, restart, readiness wait.
- Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments
and Rust doc comments updated to the Rust service.
The Test Suite job failed on every push since the black check was added
because the tree had never been formatted, and pre-commit said 120
columns while CI ran black's default 88. pyproject.toml now carries
[tool.black] / [tool.isort] (88, black profile) as the single source;
pre-commit reads it; `make format` applied it (13 files, whitespace only,
146 insertions / 128 deletions, tests unchanged at 146 passed).
ci.yml: lint (black, isort, flake8 hard errors) + pytest. The Docker
registry push, VictoriaMetrics integration test, staging/production
deploy and Apache-Bench jobs were template scaffolding for hosts and
registries that do not exist; production is a systemd unit updated by
git pull. Removed rather than left permanently skipped.
docs.yml: the "Check markdown links" step curl'd every URL in every .md
and failed on localhost examples and the Tailscale IP, and the Sphinx
jobs built artifacts nobody read. Replaced by two checks that mean
something: relative links/images in README, CONTRIBUTING and docs/
resolve inside the repo, and the FastAPI OpenAPI schema exports with
the documented endpoints present (uploaded as an artifact).
scripts/backfill_hii_waterlevel.py walks the api-v3 waterlevel_graph
endpoint (hourly wl_msl + discharge, archive back to ~2019) in full-year
windows per station and upserts into hii_waterlevel. Defaults to the
RID-mirror and key stations; --stations/--all/--start/--end/--chunk-days
override. History upserts touch only wl_msl and discharge so colliding
live-snapshot rows keep storage_percent/situation_level. Idempotent and
safe to re-run.
grabowski