grabowski
7e642555b4
Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/) collects, forecasts, notifies (ntfy + Matrix) and serves the website/API; nothing in production ran the Python service code any more. Removed (~17k lines): the FastAPI app (web_api, schemas, health_check, metrics, postgres_history, forecast_history, thaiwater), the Python RID collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL, InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/ logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling (api_parity, collector_parity, e2e_notify, install_collector.sh, gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files, sql/, the release workflow and the docs for all of that. Their tests went with them. Kept in Python: src/ml (training, evaluation harness, export; predict.py as the reference the Rust forecaster is tested against), the HII and RID-dam clients/stores for the archive backfills (their live collectors removed), the backfill scripts, retrain.sh + timer, forecast_parity.py. - PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env; the stores lost their sqlite/mysql branches (and the 2026-08 column migration). The database tests run against a throwaway schema from TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB). - The notifier settings moved from src/notify.py into src/ml/export.py; flood_config.json regenerates byte-identical. - tests/test_dashboard.py checks every path the dashboard fetches against ping-collector's router. - Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil, schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv 1.1.1 explicit (it came in through uvicorn[standard]). - Formatting contract extended to scripts/ and tests/ (reformatted once). - scripts/install.sh is the one installer (fresh host or update, in place in a checkout without touching its ownership): binary, units, uv sync, .env defaults, removes the retired water-monitor.service and any old --alert-check cron job, dry run + forecast check, restart, readiness wait. - Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments and Rust doc comments updated to the Rust service.
71 lines
2.5 KiB
Makefile
71 lines
2.5 KiB
Makefile
# Ping River Monitor. The service is Rust (rust/, ping-collector); Python is
|
|
# the model training/evaluation stack and the archive backfills (src/).
|
|
# Database tests (both sides) need TEST_DATABASE_URL, e.g.
|
|
# docker run -d --name ping-pg -e POSTGRES_PASSWORD=ping -e POSTGRES_DB=ping -p 55432:5432 postgres:16
|
|
# export TEST_DATABASE_URL=postgres://postgres:ping@localhost:55432/ping
|
|
|
|
PY_SRC := src/ scripts/ tests/ rust/forecast/testdata rust/collector/testdata
|
|
CARGO := cargo
|
|
MANIFEST := --manifest-path rust/Cargo.toml
|
|
|
|
.PHONY: help install format lint test test-py test-rust build check serve train evaluate export clean
|
|
|
|
help:
|
|
@echo "make install uv sync (Python 3.11) + pre-commit hooks"
|
|
@echo "make format black + isort (Python), cargo fmt (Rust)"
|
|
@echo "make lint the CI checks without tests (format, flake8, clippy)"
|
|
@echo "make test pytest + cargo test"
|
|
@echo "make build release build of ping-collector (rust/target/release)"
|
|
@echo "make check lint + test + the Rust 1.88 minimum-version check"
|
|
@echo "make serve website/API from the local database on 127.0.0.1:8001"
|
|
@echo "make train retrain every station's model into models/"
|
|
@echo "make evaluate rolling-origin harness on P.1 (see scripts/evaluate_variants.py)"
|
|
@echo "make export export trained models + flood_config.json for the Rust forecaster"
|
|
|
|
install:
|
|
uv sync --python 3.11
|
|
uv run pre-commit install
|
|
|
|
format:
|
|
uv run black $(PY_SRC)
|
|
uv run isort src/ scripts/ tests/
|
|
$(CARGO) fmt --all $(MANIFEST)
|
|
|
|
lint:
|
|
uv run black --check $(PY_SRC)
|
|
uv run isort --check-only src/ scripts/ tests/
|
|
uv run flake8 src/ scripts/ tests/ --count --select=E9,F63,F7,F82 --show-source --statistics
|
|
$(CARGO) fmt --all --check $(MANIFEST)
|
|
$(CARGO) clippy --all-targets --locked $(MANIFEST) -- -D warnings
|
|
|
|
test: test-py test-rust
|
|
|
|
test-py:
|
|
uv run pytest -q
|
|
|
|
test-rust:
|
|
$(CARGO) test --locked $(MANIFEST)
|
|
|
|
build:
|
|
$(CARGO) build --release --locked $(MANIFEST)
|
|
|
|
check: lint test
|
|
CARGO_TARGET_DIR=rust/target/msrv $(CARGO) +1.88 check --all-targets --locked $(MANIFEST)
|
|
|
|
serve:
|
|
rust/target/release/ping-collector serve --listen 127.0.0.1:8001 --no-analytics
|
|
|
|
train:
|
|
uv run python scripts/train_flood_model.py --stations all
|
|
|
|
evaluate:
|
|
uv run python scripts/evaluate_variants.py --stations P.1
|
|
|
|
export:
|
|
uv run python -m src.ml.export
|
|
uv run python -m src.ml.export --config
|
|
|
|
clean:
|
|
find . -type d -name "__pycache__" -prune -exec rm -rf {} +
|
|
rm -rf .pytest_cache/ htmlcov/ dist/ build/ *.egg-info/
|