grabowski
7e642555b4
Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/) collects, forecasts, notifies (ntfy + Matrix) and serves the website/API; nothing in production ran the Python service code any more. Removed (~17k lines): the FastAPI app (web_api, schemas, health_check, metrics, postgres_history, forecast_history, thaiwater), the Python RID collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL, InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/ logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling (api_parity, collector_parity, e2e_notify, install_collector.sh, gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files, sql/, the release workflow and the docs for all of that. Their tests went with them. Kept in Python: src/ml (training, evaluation harness, export; predict.py as the reference the Rust forecaster is tested against), the HII and RID-dam clients/stores for the archive backfills (their live collectors removed), the backfill scripts, retrain.sh + timer, forecast_parity.py. - PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env; the stores lost their sqlite/mysql branches (and the 2026-08 column migration). The database tests run against a throwaway schema from TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB). - The notifier settings moved from src/notify.py into src/ml/export.py; flood_config.json regenerates byte-identical. - tests/test_dashboard.py checks every path the dashboard fetches against ping-collector's router. - Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil, schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv 1.1.1 explicit (it came in through uvicorn[standard]). - Formatting contract extended to scripts/ and tests/ (reformatted once). - scripts/install.sh is the one installer (fresh host or update, in place in a checkout without touching its ownership): binary, units, uv sync, .env defaults, removes the retired water-monitor.service and any old --alert-check cron job, dry run + forecast check, restart, readiness wait. - Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments and Rust doc comments updated to the Rust service.
52 lines
2.5 KiB
Markdown
52 lines
2.5 KiB
Markdown
# Contributing
|
|
|
|
Issues and merge requests on
|
|
[git.b4l.co.th](https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor).
|
|
The repository has two halves: the service in Rust (`rust/`, see
|
|
[rust/README.md](rust/README.md)) and the model training/evaluation stack in Python
|
|
(`src/ml`, see [docs/FLOOD_FORECASTING.md](docs/FLOOD_FORECASTING.md)).
|
|
|
|
## Setup
|
|
|
|
- Python 3.11 and [uv](https://docs.astral.sh/uv/) (3.12+ has no pandas 2.0.3 wheels)
|
|
- Rust >= 1.88 from [rustup](https://rustup.rs), plus `pkg-config` and `libssl-dev` on Linux
|
|
- Docker (or any PostgreSQL 16) for the database tests
|
|
|
|
```bash
|
|
git clone https://git.b4l.co.th/B4L/Northern-Thailand-Ping-River-Monitor.git
|
|
cd Northern-Thailand-Ping-River-Monitor
|
|
make install # uv sync + pre-commit hooks
|
|
docker run -d --name ping-pg -e POSTGRES_PASSWORD=ping -e POSTGRES_DB=ping -p 55432:5432 postgres:16
|
|
export TEST_DATABASE_URL=postgres://postgres:ping@localhost:55432/ping
|
|
make check # format check, flake8, clippy, pytest, cargo test, Rust 1.88
|
|
```
|
|
|
|
Without `TEST_DATABASE_URL` the database tests are skipped, the rest still run.
|
|
|
|
## Before a merge request
|
|
|
|
- `make format`, then `make check` green. CI runs the same (`.gitea/workflows/`).
|
|
- One logical change per commit, imperative subject (`fix(collector): ...`,
|
|
`feat(web): ...`, `docs: ...`).
|
|
- Dashboard changes (`src/static/dashboard.html`): every visible string in English
|
|
and Thai (`t()` table); a screenshot in the merge request.
|
|
- API changes: keep existing fields and formats; external clients read them. New
|
|
parameters go into the route's declaration in `rust/collector/src/web/params.rs`,
|
|
which also documents them at `/docs`.
|
|
- Model changes: evaluate with `scripts/evaluate_variants.py` (rolling origin, judged
|
|
on first-alert lead and false alarms) and record the result in
|
|
`docs/FLOOD_FORECASTING.md` section 5, whether it helped or not. A feature change
|
|
must be ported to `rust/forecast` in the same change (see CLAUDE.md for the
|
|
fixture and config regeneration steps).
|
|
- Never commit credentials, `.env`, trained models (`models/*.joblib`,
|
|
`models/*.json.gz`) or the server's private addresses.
|
|
|
|
## Useful commands
|
|
|
|
```bash
|
|
make build && make serve # the website/API from your database on 127.0.0.1:8001
|
|
uv run python scripts/dev_proxy.py # the working-copy dashboard against the live API
|
|
uv run python scripts/evaluate_variants.py --stations P.1
|
|
uv run python -m src.ml.export --config # after changing thresholds or notifier settings
|
|
```
|