chore: remove the Python service; Python is the training stack only
Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/) collects, forecasts, notifies (ntfy + Matrix) and serves the website/API; nothing in production ran the Python service code any more. Removed (~17k lines): the FastAPI app (web_api, schemas, health_check, metrics, postgres_history, forecast_history, thaiwater), the Python RID collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL, InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/ logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling (api_parity, collector_parity, e2e_notify, install_collector.sh, gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files, sql/, the release workflow and the docs for all of that. Their tests went with them. Kept in Python: src/ml (training, evaluation harness, export; predict.py as the reference the Rust forecaster is tested against), the HII and RID-dam clients/stores for the archive backfills (their live collectors removed), the backfill scripts, retrain.sh + timer, forecast_parity.py. - PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env; the stores lost their sqlite/mysql branches (and the 2026-08 column migration). The database tests run against a throwaway schema from TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB). - The notifier settings moved from src/notify.py into src/ml/export.py; flood_config.json regenerates byte-identical. - tests/test_dashboard.py checks every path the dashboard fetches against ping-collector's router. - Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil, schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv 1.1.1 explicit (it came in through uvicorn[standard]). - Formatting contract extended to scripts/ and tests/ (reformatted once). - scripts/install.sh is the one installer (fresh host or update, in place in a checkout without touching its ownership): binary, units, uv sync, .env defaults, removes the retired water-monitor.service and any old --alert-check cron job, dry run + forecast check, restart, readiness wait. - Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments and Rust doc comments updated to the Rust service.
This commit is contained in:
1 parent
330fdf7804
commit
7e642555b4
125 files changed
+1300
-19558
No files matched your search
@@ -3,7 +3,7 @@
|
||||
|
||||
Two paths, both idempotent and both skipping what is already stored, so a
|
||||
rerun repairs holes left by transient failures and is safe alongside the
|
||||
hourly live collector:
|
||||
live collector (ping-collector, today + yesterday):
|
||||
|
||||
--dam-id (default: Mae Ngat) one dam, whole range, via api/dam — a handful
|
||||
of requests for the entire 2009-today archive
|
||||
@@ -26,13 +26,8 @@ import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
from src.config import Config
|
||||
from src.rid_reservoir import (
|
||||
MAE_NGAT_DAM_ID,
|
||||
RidReservoirStore,
|
||||
backfill,
|
||||
backfill_dam,
|
||||
)
|
||||
from src.config import database_url
|
||||
from src.rid_reservoir import MAE_NGAT_DAM_ID, RidReservoirStore, backfill, backfill_dam
|
||||
|
||||
DEFAULT_START = datetime.date(2018, 8, 1) # start of the water_measurements grid
|
||||
|
||||
@@ -71,18 +66,14 @@ def main(argv=None) -> int:
|
||||
logging.basicConfig(
|
||||
level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s"
|
||||
)
|
||||
if args.db_url:
|
||||
# postgresql+psycopg2://... -> postgresql (driver suffix is not a dialect)
|
||||
connection_string = args.db_url
|
||||
db_type = args.db_url.split(":", 1)[0].split("+", 1)[0]
|
||||
else:
|
||||
cfg = Config.get_database_config()
|
||||
if cfg["type"] not in ("sqlite", "postgresql", "mysql"):
|
||||
print(f"requires a SQL database, got {cfg['type']}", file=sys.stderr)
|
||||
return 1
|
||||
connection_string, db_type = cfg["connection_string"], cfg["type"]
|
||||
connection_string = args.db_url or database_url()
|
||||
if not connection_string:
|
||||
print(
|
||||
"no database: set POSTGRES_CONNECTION_STRING or POSTGRES_*", file=sys.stderr
|
||||
)
|
||||
return 1
|
||||
|
||||
store = RidReservoirStore(connection_string, db_type)
|
||||
store = RidReservoirStore(connection_string)
|
||||
if not store.connect():
|
||||
print(
|
||||
"database connection failed — check the connection string",
|
||||
|
||||
Reference in new issue
Block a user