chore: remove the Python service; Python is the training stack only

Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/)
collects, forecasts, notifies (ntfy + Matrix) and serves the website/API;
nothing in production ran the Python service code any more.

Removed (~17k lines): the FastAPI app (web_api, schemas, health_check,
metrics, postgres_history, forecast_history, thaiwater), the Python RID
collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix
alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL,
InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/
logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling
(api_parity, collector_parity, e2e_notify, install_collector.sh,
gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files,
sql/, the release workflow and the docs for all of that. Their tests went
with them.

Kept in Python: src/ml (training, evaluation harness, export; predict.py as
the reference the Rust forecaster is tested against), the HII and RID-dam
clients/stores for the archive backfills (their live collectors removed),
the backfill scripts, retrain.sh + timer, forecast_parity.py.

- PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env;
  the stores lost their sqlite/mysql branches (and the 2026-08 column
  migration). The database tests run against a throwaway schema from
  TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the
  Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB).
- The notifier settings moved from src/notify.py into src/ml/export.py;
  flood_config.json regenerates byte-identical.
- tests/test_dashboard.py checks every path the dashboard fetches against
  ping-collector's router.
- Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil,
  schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv
  1.1.1 explicit (it came in through uvicorn[standard]).
- Formatting contract extended to scripts/ and tests/ (reformatted once).
- scripts/install.sh is the one installer (fresh host or update, in place in
  a checkout without touching its ownership): binary, units, uv sync, .env
  defaults, removes the retired water-monitor.service and any old
  --alert-check cron job, dry run + forecast check, restart, readiness wait.
- Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments
  and Rust doc comments updated to the Rust service.
This commit is contained in:
grabowski committed 2026-10-18 17:15:07 +07:00
1 parent 330fdf7804
commit 7e642555b4
125 files changed
+1300 -19558

No files matched your search

+46 -13
View File
@@ -1,23 +1,53 @@
import re
from pathlib import Path
DASHBOARD_PATH = Path(__file__).parents[1] / "src" / "static" / "dashboard.html"
ROUTER_PATH = (
Path(__file__).parents[1] / "rust" / "collector" / "src" / "web" / "mod.rs"
)
def _rust_routes() -> set:
"""Paths the website serves (ping-collector's axum router)."""
source = ROUTER_PATH.read_text(encoding="utf-8")
return set(re.findall(r'\.route\(\s*"([^"]+)"', source))
def test_every_api_path_the_dashboard_fetches_is_served():
html = DASHBOARD_PATH.read_text(encoding="utf-8")
routes = _rust_routes()
patterns = [
re.compile("^" + re.sub(r"\\\{[^}]+\\\}", "[^/]+", re.escape(r)) + "$")
for r in routes
]
fetched = set(re.findall(r"fetch\(\s*[`'\"](/[^`'\"?$]*)", html))
assert fetched, "no fetch() calls found"
missing = [
path
for path in fetched
if not path.startswith("/static/")
and not any(p.match(path.rstrip("/") or "/") for p in patterns)
and not any(r.startswith(path) for r in routes)
]
assert not missing, f"dashboard fetches paths the server does not route: {missing}"
def test_dashboard_contains_live_map_and_flow_visualization():
html = DASHBOARD_PATH.read_text(encoding="utf-8")
assert "id=\"station-map\"" in html
assert "id=\"river-flow\"" in html
assert 'id="station-map"' in html
assert 'id="river-flow"' in html
assert "fetch('/stations')" in html or 'fetch("/stations")' in html
assert "fetch('/measurements/latest" in html or 'fetch(\"/measurements/latest' in html
assert (
"fetch('/measurements/latest" in html or 'fetch("/measurements/latest' in html
)
assert "leaflet" in html.lower()
def _body(html: str) -> str:
"""Markup only. The STRINGS table repeats every English phrase, so a
whole-file search would pass even if an element were deleted."""
return html[html.index("<body>"):html.index("<script src=")]
return html[html.index("<body>") : html.index("<script src=")]
def test_dashboard_explains_flow_legend_and_refresh():
@@ -57,13 +87,14 @@ def test_dashboard_has_seo_and_indexing_files():
robots = (static_dir / "robots.txt").read_text(encoding="utf-8")
assert "Sitemap: https://water.buildfor.life/sitemap.xml" in robots
assert "<loc>https://water.buildfor.life/</loc>" in (static_dir / "sitemap.xml").read_text(encoding="utf-8")
assert "Ping River Live Monitor" in (static_dir / "llms.txt").read_text(encoding="utf-8")
assert "<loc>https://water.buildfor.life/</loc>" in (
static_dir / "sitemap.xml"
).read_text(encoding="utf-8")
assert "Ping River Live Monitor" in (static_dir / "llms.txt").read_text(
encoding="utf-8"
)
from src import web_api
routes = {route.path for route in web_api.app.routes}
assert {"/robots.txt", "/llms.txt", "/sitemap.xml"} <= routes
assert {"/robots.txt", "/llms.txt", "/sitemap.xml"} <= _rust_routes()
def test_dashboard_shows_hii_rainfall_layer():
@@ -124,7 +155,9 @@ def test_dashboard_translations_cover_both_languages():
missing_th = tables["en"] - tables["th"]
missing_en = tables["th"] - tables["en"]
assert not missing_th, f"keys missing a Thai translation: {sorted(missing_th)}"
assert not missing_en, f"Thai-only keys with no English fallback: {sorted(missing_en)}"
assert (
not missing_en
), f"Thai-only keys with no English fallback: {sorted(missing_en)}"
def test_dashboard_i18n_markup_keys_exist():
@@ -156,7 +189,7 @@ def test_dashboard_translates_user_visible_aria_labels():
body = _body(html)
for match in re.finditer(r'<[^>]*\saria-label="[^"]+"[^>]*>', body):
tag = match.group(0)
if "data-i18n-aria" in tag or "id=\"lang-toggle\"" in tag:
if "data-i18n-aria" in tag or 'id="lang-toggle"' in tag:
continue # the toggle sets its own label per language in JS
raise AssertionError(f"aria-label without a translation key: {tag[:120]}")