chore: remove the Python service; Python is the training stack only

Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/)
collects, forecasts, notifies (ntfy + Matrix) and serves the website/API;
nothing in production ran the Python service code any more.

Removed (~17k lines): the FastAPI app (web_api, schemas, health_check,
metrics, postgres_history, forecast_history, thaiwater), the Python RID
collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix
alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL,
InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/
logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling
(api_parity, collector_parity, e2e_notify, install_collector.sh,
gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files,
sql/, the release workflow and the docs for all of that. Their tests went
with them.

Kept in Python: src/ml (training, evaluation harness, export; predict.py as
the reference the Rust forecaster is tested against), the HII and RID-dam
clients/stores for the archive backfills (their live collectors removed),
the backfill scripts, retrain.sh + timer, forecast_parity.py.

- PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env;
  the stores lost their sqlite/mysql branches (and the 2026-08 column
  migration). The database tests run against a throwaway schema from
  TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the
  Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB).
- The notifier settings moved from src/notify.py into src/ml/export.py;
  flood_config.json regenerates byte-identical.
- tests/test_dashboard.py checks every path the dashboard fetches against
  ping-collector's router.
- Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil,
  schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv
  1.1.1 explicit (it came in through uvicorn[standard]).
- Formatting contract extended to scripts/ and tests/ (reformatted once).
- scripts/install.sh is the one installer (fresh host or update, in place in
  a checkout without touching its ownership): binary, units, uv sync, .env
  defaults, removes the retired water-monitor.service and any old
  --alert-check cron job, dry run + forecast check, restart, readiness wait.
- Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments
  and Rust doc comments updated to the Rust service.
This commit is contained in:
grabowski committed 2026-10-18 17:15:07 +07:00
1 parent 330fdf7804
commit 7e642555b4
125 files changed
+1300 -19558

No files matched your search

+9 -13
View File
@@ -2,7 +2,7 @@
"""Backfill the openmeteo_rain table with the full 2021+ catchment history.
Usage:
uv run scripts/backfill_rain_db.py # DB from Config/.env
uv run scripts/backfill_rain_db.py # DB from POSTGRES_* / .env
uv run scripts/backfill_rain_db.py --db-url postgresql://...
"""
@@ -13,7 +13,7 @@ import sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.config import Config
from src.config import database_url
from src.ml.rain import backfill_db
@@ -25,21 +25,17 @@ def main(argv=None) -> int:
logging.basicConfig(
level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s"
)
if args.db_url:
# postgresql+psycopg2://... -> postgresql (driver suffix is not a dialect)
connection_string = args.db_url
db_type = args.db_url.split(":", 1)[0].split("+", 1)[0]
else:
cfg = Config.get_database_config()
if cfg["type"] not in ("sqlite", "postgresql", "mysql"):
print(f"requires a SQL database, got {cfg['type']}", file=sys.stderr)
return 1
connection_string, db_type = cfg["connection_string"], cfg["type"]
connection_string = args.db_url or database_url()
if not connection_string:
print(
"no database: set POSTGRES_CONNECTION_STRING or POSTGRES_*", file=sys.stderr
)
return 1
from sqlalchemy import create_engine
engine = create_engine(connection_string, pool_pre_ping=True)
saved = backfill_db(engine, db_type)
saved = backfill_db(engine)
print(f"backfilled {saved} hourly rows into openmeteo_rain")
return 0 if saved else 1