chore: remove the Python service; Python is the training stack only

Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/)
collects, forecasts, notifies (ntfy + Matrix) and serves the website/API;
nothing in production ran the Python service code any more.

Removed (~17k lines): the FastAPI app (web_api, schemas, health_check,
metrics, postgres_history, forecast_history, thaiwater), the Python RID
collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix
alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL,
InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/
logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling
(api_parity, collector_parity, e2e_notify, install_collector.sh,
gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files,
sql/, the release workflow and the docs for all of that. Their tests went
with them.

Kept in Python: src/ml (training, evaluation harness, export; predict.py as
the reference the Rust forecaster is tested against), the HII and RID-dam
clients/stores for the archive backfills (their live collectors removed),
the backfill scripts, retrain.sh + timer, forecast_parity.py.

- PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env;
  the stores lost their sqlite/mysql branches (and the 2026-08 column
  migration). The database tests run against a throwaway schema from
  TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the
  Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB).
- The notifier settings moved from src/notify.py into src/ml/export.py;
  flood_config.json regenerates byte-identical.
- tests/test_dashboard.py checks every path the dashboard fetches against
  ping-collector's router.
- Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil,
  schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv
  1.1.1 explicit (it came in through uvicorn[standard]).
- Formatting contract extended to scripts/ and tests/ (reformatted once).
- scripts/install.sh is the one installer (fresh host or update, in place in
  a checkout without touching its ownership): binary, units, uv sync, .env
  defaults, removes the retired water-monitor.service and any old
  --alert-check cron job, dry run + forecast check, restart, readiness wait.
- Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments
  and Rust doc comments updated to the Rust service.
This commit is contained in:
grabowski committed 2026-10-18 17:15:07 +07:00
1 parent 330fdf7804
commit 7e642555b4
125 files changed
+1300 -19558

No files matched your search

+25 -15
View File
@@ -1,13 +1,13 @@
name: CI
# What this checks, on every push and PR to master:
# The Python side (src/ml training/evaluation stack, archive backfills), on
# every push and PR to master:
# 1. formatting contract (black + isort, config in pyproject.toml)
# 2. flake8 hard-error gate (syntax, undefined names)
# 3. the pytest suite (synthetic data, no DB/network; ~1 min)
# Docker build / staging / production / perf jobs from the original template
# were removed: there is no registry, no staging host, and production is a
# systemd unit deployed by `git pull` on the server (docs/FLOOD_FORECASTING.md
# section 6, scripts/install.sh). Re-add a job when the thing it deploys exists.
# 3. the pytest suite: synthetic data, no network; the database tests run
# against a throwaway PostgreSQL service (TEST_DATABASE_URL, as in rust.yml)
# The service itself (ping-collector) is checked by rust.yml. Production is a
# systemd unit deployed by `git pull` + scripts/install.sh on the server.
on:
push:
@@ -15,7 +15,7 @@ on:
pull_request:
branches: [master]
schedule:
# daily, catches dependency drift / upstream API changes in the tests
# daily, catches dependency drift
- cron: "0 2 * * *"
workflow_dispatch:
@@ -41,17 +41,30 @@ jobs:
pip install --root-user-action=ignore black==26.5.1 isort==5.12.0 flake8==6.1.0
- name: black
run: black --check --diff src/ *.py
run: black --check --diff src/ scripts/ tests/ rust/forecast/testdata rust/collector/testdata
- name: isort
run: isort --check-only --diff src/ *.py
run: isort --check-only --diff src/ scripts/ tests/
- name: flake8 (errors only)
run: flake8 src/ --count --select=E9,F63,F7,F82 --show-source --statistics
run: flake8 src/ scripts/ tests/ --count --select=E9,F63,F7,F82 --show-source --statistics
test:
name: Test suite
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16
env:
POSTGRES_PASSWORD: ping
POSTGRES_DB: ping
options: >-
--health-cmd pg_isready
--health-interval 5s
--health-timeout 5s
--health-retries 20
env:
TEST_DATABASE_URL: postgres://postgres:ping@postgres:5432/ping
steps:
- uses: actions/checkout@v4
@@ -66,10 +79,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip --root-user-action=ignore
pip install --root-user-action=ignore -r requirements.txt
pip install --root-user-action=ignore pytest==9.1.1 pytest-asyncio==0.21.1
pip install --root-user-action=ignore -r requirements.txt pytest==9.1.1
- name: pytest
env:
DB_TYPE: sqlite
run: pytest -q -p no:cacheprovider
run: pytest -q -p no:cacheprovider -rs