B4L/Northern-Thailand-Ping-River-Monitor

0 Fork 0

Files

T

grabowski 7e642555b4 chore: remove the Python service; Python is the training stack only

Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/)
collects, forecasts, notifies (ntfy + Matrix) and serves the website/API;
nothing in production ran the Python service code any more.

Removed (~17k lines): the FastAPI app (web_api, schemas, health_check,
metrics, postgres_history, forecast_history, thaiwater), the Python RID
collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix
alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL,
InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/
logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling
(api_parity, collector_parity, e2e_notify, install_collector.sh,
gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files,
sql/, the release workflow and the docs for all of that. Their tests went
with them.

Kept in Python: src/ml (training, evaluation harness, export; predict.py as
the reference the Rust forecaster is tested against), the HII and RID-dam
clients/stores for the archive backfills (their live collectors removed),
the backfill scripts, retrain.sh + timer, forecast_parity.py.

- PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env;
  the stores lost their sqlite/mysql branches (and the 2026-08 column
  migration). The database tests run against a throwaway schema from
  TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the
  Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB).
- The notifier settings moved from src/notify.py into src/ml/export.py;
  flood_config.json regenerates byte-identical.
- tests/test_dashboard.py checks every path the dashboard fetches against
  ping-collector's router.
- Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil,
  schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv
  1.1.1 explicit (it came in through uvicorn[standard]).
- Formatting contract extended to scripts/ and tests/ (reformatted once).
- scripts/install.sh is the one installer (fresh host or update, in place in
  a checkout without touching its ownership): binary, units, uv sync, .env
  defaults, removes the retired water-monitor.service and any old
  --alert-check cron job, dry run + forecast check, restart, readiness wait.
- Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments
  and Rust doc comments updated to the Rust service.

2026-10-18 17:15:07 +07:00

2.8 KiB

Raw Blame History

Quick Matrix Alerting Setup

Step 1: Get Matrix Account

Go to https://app.element.io or install Element app
Create account or login with existing Matrix account

Step 2: Get Access Token

Method 1: Element Web (Recommended)

Open Element in browser: https://app.element.io
Login to your account
Click Settings (gear icon) → Help & About → Advanced
Copy your "Access Token" (starts with syt_... or similar)

Method 2: Command Line

curl -X POST https://matrix.org/_matrix/client/v3/login \
  -H "Content-Type: application/json" \
  -d '{
    "type": "m.login.password",
    "user": "your_username",
    "password": "your_password"
  }'

Step 3: Create Alert Room

In Element, click "+" to create new room
Name: "Water Level Alerts"
Set to Private
Copy the room ID from room settings (format: !roomid:matrix.org)

Step 4: Configure .env File

Add these to your .env file:

MATRIX_HOMESERVER=https://matrix.org
MATRIX_ACCESS_TOKEN=syt_your_access_token_here
MATRIX_ROOM_ID=!your_room_id:matrix.org

Step 5: Restart and check

The Matrix messages are sent by ping-collector (the Rust service, with FORECAST_IN_COLLECTOR=true), together with the ntfy notifications:

sudo systemctl restart ping-collector
journalctl -u ping-collector | grep -i matrix      # "Matrix notifications: room ... on ..."
# what would be sent right now (ntfy and Matrix), changing nothing:
sudo -u water-monitor /opt/thailand-water-monitor/bin/ping-collector notify --dry-run

What the room receives

The same transitions as the ntfy topics (docs/NOTIFICATIONS.md), each once: a station crossing its warning or danger level and back (0.10 m hysteresis), the P.1 early-warning outlook turning on and off, and the gauge feed going stale or recovering. ntfy's per-topic summaries ("Ping River: warning level reached") are not repeated in the room. For example:

⚠️ **Warning level at P.1**
Nawarat Bridge, Chiang Mai city: 3.75 m at 18 Oct 14:00, above the warning level of 3.70 m.

📈 Dashboard: https://water.buildfor.life/

The room has its own delivery state (matrix: keys in notification_state), so a Matrix outage is retried on Matrix only and does not re-send ntfy.

The old run.py --alert-check cron job (every 15 minutes, repeating an alert for as long as a level stayed high) is replaced by this; scripts/install.sh comments it out of the crontab if it is still there, keeping a backup next to the app.

Troubleshooting

HTTP 401/403 in journalctl -u ping-collector: the access token is invalid or the account is not in the room.
Room not found: the room ID needs the ! prefix and the :homeserver suffix.
Nothing sent: there was no transition; ping-collector notify --dry-run shows the readings judged and what would be sent.

2.8 KiB Raw Blame History