chore: remove the Python service; Python is the training stack only
Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/) collects, forecasts, notifies (ntfy + Matrix) and serves the website/API; nothing in production ran the Python service code any more. Removed (~17k lines): the FastAPI app (web_api, schemas, health_check, metrics, postgres_history, forecast_history, thaiwater), the Python RID collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL, InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/ logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling (api_parity, collector_parity, e2e_notify, install_collector.sh, gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files, sql/, the release workflow and the docs for all of that. Their tests went with them. Kept in Python: src/ml (training, evaluation harness, export; predict.py as the reference the Rust forecaster is tested against), the HII and RID-dam clients/stores for the archive backfills (their live collectors removed), the backfill scripts, retrain.sh + timer, forecast_parity.py. - PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env; the stores lost their sqlite/mysql branches (and the 2026-08 column migration). The database tests run against a throwaway schema from TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB). - The notifier settings moved from src/notify.py into src/ml/export.py; flood_config.json regenerates byte-identical. - tests/test_dashboard.py checks every path the dashboard fetches against ping-collector's router. - Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil, schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv 1.1.1 explicit (it came in through uvicorn[standard]). - Formatting contract extended to scripts/ and tests/ (reformatted once). - scripts/install.sh is the one installer (fresh host or update, in place in a checkout without touching its ownership): binary, units, uv sync, .env defaults, removes the retired water-monitor.service and any old --alert-check cron job, dry run + forecast check, restart, readiness wait. - Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments and Rust doc comments updated to the Rust service.
This commit is contained in:
1 parent
330fdf7804
commit
7e642555b4
125 files changed
+1300
-19558
No files matched your search
+25
-15
@@ -1,13 +1,13 @@
|
||||
name: CI
|
||||
|
||||
# What this checks, on every push and PR to master:
|
||||
# The Python side (src/ml training/evaluation stack, archive backfills), on
|
||||
# every push and PR to master:
|
||||
# 1. formatting contract (black + isort, config in pyproject.toml)
|
||||
# 2. flake8 hard-error gate (syntax, undefined names)
|
||||
# 3. the pytest suite (synthetic data, no DB/network; ~1 min)
|
||||
# Docker build / staging / production / perf jobs from the original template
|
||||
# were removed: there is no registry, no staging host, and production is a
|
||||
# systemd unit deployed by `git pull` on the server (docs/FLOOD_FORECASTING.md
|
||||
# section 6, scripts/install.sh). Re-add a job when the thing it deploys exists.
|
||||
# 3. the pytest suite: synthetic data, no network; the database tests run
|
||||
# against a throwaway PostgreSQL service (TEST_DATABASE_URL, as in rust.yml)
|
||||
# The service itself (ping-collector) is checked by rust.yml. Production is a
|
||||
# systemd unit deployed by `git pull` + scripts/install.sh on the server.
|
||||
|
||||
on:
|
||||
push:
|
||||
@@ -15,7 +15,7 @@ on:
|
||||
pull_request:
|
||||
branches: [master]
|
||||
schedule:
|
||||
# daily, catches dependency drift / upstream API changes in the tests
|
||||
# daily, catches dependency drift
|
||||
- cron: "0 2 * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
@@ -41,17 +41,30 @@ jobs:
|
||||
pip install --root-user-action=ignore black==26.5.1 isort==5.12.0 flake8==6.1.0
|
||||
|
||||
- name: black
|
||||
run: black --check --diff src/ *.py
|
||||
run: black --check --diff src/ scripts/ tests/ rust/forecast/testdata rust/collector/testdata
|
||||
|
||||
- name: isort
|
||||
run: isort --check-only --diff src/ *.py
|
||||
run: isort --check-only --diff src/ scripts/ tests/
|
||||
|
||||
- name: flake8 (errors only)
|
||||
run: flake8 src/ --count --select=E9,F63,F7,F82 --show-source --statistics
|
||||
run: flake8 src/ scripts/ tests/ --count --select=E9,F63,F7,F82 --show-source --statistics
|
||||
|
||||
test:
|
||||
name: Test suite
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env:
|
||||
POSTGRES_PASSWORD: ping
|
||||
POSTGRES_DB: ping
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 5s
|
||||
--health-timeout 5s
|
||||
--health-retries 20
|
||||
env:
|
||||
TEST_DATABASE_URL: postgres://postgres:ping@postgres:5432/ping
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
@@ -66,10 +79,7 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip --root-user-action=ignore
|
||||
pip install --root-user-action=ignore -r requirements.txt
|
||||
pip install --root-user-action=ignore pytest==9.1.1 pytest-asyncio==0.21.1
|
||||
pip install --root-user-action=ignore -r requirements.txt pytest==9.1.1
|
||||
|
||||
- name: pytest
|
||||
env:
|
||||
DB_TYPE: sqlite
|
||||
run: pytest -q -p no:cacheprovider
|
||||
run: pytest -q -p no:cacheprovider -rs
|
||||
Reference in new issue
Block a user