grabowski
4e619aa2ae
CI / Format & lint (push) Successful in 19s
CI / Test suite (push) Successful in 47s
Docs / Validate documentation (push) Successful in 19s
Rust / fmt, clippy, test, build (push) Successful in 7m35s
Security / Dependency vulnerabilities (push) Successful in 1m37s
Security / Static analysis (push) Successful in 17s
Security / License report (push) Successful in 1m30s
rust/collector replaces the collection half of the Python web app: RID hourly gauges (with the late-report semantics: store everything, retry every 10 min while RID is behind, re-fetch yesterday's holes, startup gap fill), HII rain + water level, RID dams (today + yesterday) and Open-Meteo rain. Same tables and upsert semantics, one UNNEST statement per table, Bangkok wall-clock time regardless of the server time zone. Python side: COLLECTION_ENABLED=false makes the API leader stop collecting and instead re-forecast + notify when the newest RID reading in the database advances (polled every FORECAST_POLL_SECONDS) and once per clock hour; /scraping/status then serves the collector's status file; /scrape/trigger answers 409. Verified: 20 Rust tests (parsers, gap/schedule logic, database tests against PostgreSQL); live parity run of both collectors into two empty databases: all 9 tables row-for-row identical (scripts/collector_parity.py); the Python API served health, measurements, stats, status and model forecasts from a database filled only by ping-collector. 177 Python tests pass. reqwest uses native-tls: app.rid.go.th offers only TLS 1.2 CBC suites, which rustls refuses. Deploy/cutover/rollback: scripts/install_collector.sh, scripts/ping-collector.service; CI: .gitea/workflows/rust.yml (fmt, clippy, tests with a PostgreSQL service, release artifact).
48 lines
2.7 KiB
Markdown
48 lines
2.7 KiB
Markdown
# CLAUDE.md
|
|
|
|
Guidance for AI coding agents working in this repository.
|
|
|
|
## What this is
|
|
|
|
Flood monitoring and forecasting for the Ping River, Chiang Mai. Public dashboard and
|
|
API at https://water.buildfor.life/ (never publish the server's private/Tailscale IP).
|
|
Production: one systemd unit on a small VPS, `/opt/thailand-water-monitor`, user
|
|
`water-monitor`, interpreter `.venv/bin/python` (uv-managed), updated by `git pull`.
|
|
|
|
## Rules
|
|
|
|
- Python 3.11 only. `uv sync --python 3.11`; run everything as `uv run ...`.
|
|
- `make format` (black 88 / isort black profile, config in pyproject.toml) before
|
|
committing; CI fails on formatting. `make test` must stay green — tests are
|
|
synthetic-data only, never add one that needs the DB or network.
|
|
- Timestamps everywhere are Asia/Bangkok wall-clock with no offset. The dashboard
|
|
parses them with `parseTs()` and renders with `timeZone: TZ`; keep it that way.
|
|
- Model changes go through the rolling-origin harness (`scripts/evaluate_variants.py`)
|
|
and are judged on first-alert LEAD and false alarms, not MAE. Record results, positive
|
|
or negative, in `docs/FLOOD_FORECASTING.md` section 5. Do not change what is deployed
|
|
(`rise_rain` / hgb-v3) without a harness result that beats it on lead.
|
|
- `train_all()` must never silently produce a gauge-only (v2) model; the guard that
|
|
raises `RainUnavailableError` stays.
|
|
- No `git add -A`: zero-byte shell-accident files (`#`, `$(wc`, ...) have been committed
|
|
before. Stage files by name.
|
|
- Do not add Co-Authored-By trailers.
|
|
- The dashboard is a single file, `src/static/dashboard.html`, EN + TH via the `t()`
|
|
table: every user-visible string needs both languages.
|
|
- Rust (`rust/`, see rust/README.md): `cargo fmt --all`, `cargo clippy --all-targets
|
|
-- -D warnings`, `cargo test` (database tests need `TEST_DATABASE_URL`). The Rust
|
|
collector writes the same tables as the Python collectors: a change to a table,
|
|
upsert or parser on one side must be made on the other until the Python side is
|
|
retired (check with a live run of both into two databases).
|
|
|
|
## Where things are
|
|
|
|
- `src/web_api.py` FastAPI app; `src/water_scraper_v3.py` RID collector;
|
|
`src/hii_collector.py` ThaiWater/HII; `src/ml/` features/train/evaluate/predict,
|
|
`rain.py` (Open-Meteo), `dam.py`, `hii_rain.py`.
|
|
- `scripts/retrain.sh` + `water-monitor-retrain.timer`: monthly retrain with staged
|
|
promote. `scripts/dev_proxy.py`: serve the working-copy dashboard against the live API.
|
|
- `rust/collector` (`ping-collector`): Rust data collector; with it deployed the
|
|
Python app runs `COLLECTION_ENABLED=false`. Rewrite plan: `docs/RUST_REWRITE.md`.
|
|
- `docs/FLOOD_FORECASTING.md` is the authoritative model write-up; `docs/DATA_SOURCES.md`
|
|
the source catalog.
|