chore: remove the Python service; Python is the training stack only
Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/) collects, forecasts, notifies (ntfy + Matrix) and serves the website/API; nothing in production ran the Python service code any more. Removed (~17k lines): the FastAPI app (web_api, schemas, health_check, metrics, postgres_history, forecast_history, thaiwater), the Python RID collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL, InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/ logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling (api_parity, collector_parity, e2e_notify, install_collector.sh, gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files, sql/, the release workflow and the docs for all of that. Their tests went with them. Kept in Python: src/ml (training, evaluation harness, export; predict.py as the reference the Rust forecaster is tested against), the HII and RID-dam clients/stores for the archive backfills (their live collectors removed), the backfill scripts, retrain.sh + timer, forecast_parity.py. - PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env; the stores lost their sqlite/mysql branches (and the 2026-08 column migration). The database tests run against a throwaway schema from TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB). - The notifier settings moved from src/notify.py into src/ml/export.py; flood_config.json regenerates byte-identical. - tests/test_dashboard.py checks every path the dashboard fetches against ping-collector's router. - Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil, schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv 1.1.1 explicit (it came in through uvicorn[standard]). - Formatting contract extended to scripts/ and tests/ (reformatted once). - scripts/install.sh is the one installer (fresh host or update, in place in a checkout without touching its ownership): binary, units, uv sync, .env defaults, removes the retired water-monitor.service and any old --alert-check cron job, dry run + forecast check, restart, readiness wait. - Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments and Rust doc comments updated to the Rust service.
This commit is contained in:
1 parent
330fdf7804
commit
7e642555b4
125 files changed
+1300
-19558
No files matched your search
@@ -51,7 +51,7 @@ class TestHourlyFrame:
|
||||
|
||||
|
||||
class TestLoadDaily:
|
||||
def test_empty_db_result_does_not_wipe_cache(self, tmp_path):
|
||||
def test_empty_db_result_does_not_wipe_cache(self, tmp_path, pg_url):
|
||||
from sqlalchemy import create_engine, text
|
||||
|
||||
from src.ml.dam import CACHE_FILE, load_daily
|
||||
@@ -60,15 +60,16 @@ class TestLoadDaily:
|
||||
cache_path = tmp_path / CACHE_FILE
|
||||
_daily(3).rename_axis("date").to_csv(cache_path, compression="gzip")
|
||||
# Reachable DB whose table exists but is empty
|
||||
db = f"sqlite:///{tmp_path}/empty_dam.db"
|
||||
with create_engine(db).begin() as conn:
|
||||
engine = create_engine(pg_url)
|
||||
with engine.begin() as conn:
|
||||
conn.execute(
|
||||
text(
|
||||
"CREATE TABLE rid_reservoir_daily (dam_id TEXT, date DATE, "
|
||||
"storage_pct REAL, inflow_mcm REAL, outflow_mcm REAL)"
|
||||
)
|
||||
)
|
||||
result = load_daily(db_url=db, cache_dir=tmp_path)
|
||||
engine.dispose()
|
||||
result = load_daily(db_url=pg_url, cache_dir=tmp_path)
|
||||
assert result.empty # honest empty result...
|
||||
cached = pd.read_csv(cache_path, index_col=0)
|
||||
assert len(cached) == 3 # ...but the good cache survives
|
||||
@@ -122,7 +123,8 @@ class TestFeatureGating:
|
||||
dam = hourly_frame(_daily(days=20, start="2024-09-10"))
|
||||
X = features.build_features(grid, "P.1", dam=dam)
|
||||
ts = pd.Timestamp("2024-09-24 12:00")
|
||||
expected = X.loc[ts, "dam_storage_pct"] - X.loc[
|
||||
ts - pd.Timedelta(hours=72), "dam_storage_pct"
|
||||
]
|
||||
expected = (
|
||||
X.loc[ts, "dam_storage_pct"]
|
||||
- X.loc[ts - pd.Timedelta(hours=72), "dam_storage_pct"]
|
||||
)
|
||||
assert abs(X.loc[ts, "dam_storage_pct_d3"] - expected) < 1e-9
|
||||
Reference in new issue
Block a user