grabowski
4e619aa2ae
CI / Format & lint (push) Successful in 19s
CI / Test suite (push) Successful in 47s
Docs / Validate documentation (push) Successful in 19s
Rust / fmt, clippy, test, build (push) Successful in 7m35s
Security / Dependency vulnerabilities (push) Successful in 1m37s
Security / Static analysis (push) Successful in 17s
Security / License report (push) Successful in 1m30s
rust/collector replaces the collection half of the Python web app: RID hourly gauges (with the late-report semantics: store everything, retry every 10 min while RID is behind, re-fetch yesterday's holes, startup gap fill), HII rain + water level, RID dams (today + yesterday) and Open-Meteo rain. Same tables and upsert semantics, one UNNEST statement per table, Bangkok wall-clock time regardless of the server time zone. Python side: COLLECTION_ENABLED=false makes the API leader stop collecting and instead re-forecast + notify when the newest RID reading in the database advances (polled every FORECAST_POLL_SECONDS) and once per clock hour; /scraping/status then serves the collector's status file; /scrape/trigger answers 409. Verified: 20 Rust tests (parsers, gap/schedule logic, database tests against PostgreSQL); live parity run of both collectors into two empty databases: all 9 tables row-for-row identical (scripts/collector_parity.py); the Python API served health, measurements, stats, status and model forecasts from a database filled only by ping-collector. 177 Python tests pass. reqwest uses native-tls: app.rid.go.th offers only TLS 1.2 CBC suites, which rustls refuses. Deploy/cutover/rollback: scripts/install_collector.sh, scripts/ping-collector.service; CI: .gitea/workflows/rust.yml (fmt, clippy, tests with a PostgreSQL service, release artifact).
2.7 KiB
2.7 KiB
CLAUDE.md
Guidance for AI coding agents working in this repository.
What this is
Flood monitoring and forecasting for the Ping River, Chiang Mai. Public dashboard and
API at https://water.buildfor.life/ (never publish the server's private/Tailscale IP).
Production: one systemd unit on a small VPS, /opt/thailand-water-monitor, user
water-monitor, interpreter .venv/bin/python (uv-managed), updated by git pull.
Rules
- Python 3.11 only.
uv sync --python 3.11; run everything asuv run .... make format(black 88 / isort black profile, config in pyproject.toml) before committing; CI fails on formatting.make testmust stay green — tests are synthetic-data only, never add one that needs the DB or network.- Timestamps everywhere are Asia/Bangkok wall-clock with no offset. The dashboard
parses them with
parseTs()and renders withtimeZone: TZ; keep it that way. - Model changes go through the rolling-origin harness (
scripts/evaluate_variants.py) and are judged on first-alert LEAD and false alarms, not MAE. Record results, positive or negative, indocs/FLOOD_FORECASTING.mdsection 5. Do not change what is deployed (rise_rain/ hgb-v3) without a harness result that beats it on lead. train_all()must never silently produce a gauge-only (v2) model; the guard that raisesRainUnavailableErrorstays.- No
git add -A: zero-byte shell-accident files (#,$(wc, ...) have been committed before. Stage files by name. - Do not add Co-Authored-By trailers.
- The dashboard is a single file,
src/static/dashboard.html, EN + TH via thet()table: every user-visible string needs both languages. - Rust (
rust/, see rust/README.md):cargo fmt --all,cargo clippy --all-targets -- -D warnings,cargo test(database tests needTEST_DATABASE_URL). The Rust collector writes the same tables as the Python collectors: a change to a table, upsert or parser on one side must be made on the other until the Python side is retired (check with a live run of both into two databases).
Where things are
src/web_api.pyFastAPI app;src/water_scraper_v3.pyRID collector;src/hii_collector.pyThaiWater/HII;src/ml/features/train/evaluate/predict,rain.py(Open-Meteo),dam.py,hii_rain.py.scripts/retrain.sh+water-monitor-retrain.timer: monthly retrain with staged promote.scripts/dev_proxy.py: serve the working-copy dashboard against the live API.rust/collector(ping-collector): Rust data collector; with it deployed the Python app runsCOLLECTION_ENABLED=false. Rewrite plan:docs/RUST_REWRITE.md.docs/FLOOD_FORECASTING.mdis the authoritative model write-up;docs/DATA_SOURCES.mdthe source catalog.