Now a direct dependency, pip-audit sees it; it used to come in through uvicorn. The .env parser is unchanged: rust/collector/testdata/gen_envfile_cases.py regenerates the 614-case fixture byte-identical under 1.2.2, so envfile.rs still matches.
Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/)
collects, forecasts, notifies (ntfy + Matrix) and serves the website/API;
nothing in production ran the Python service code any more.
Removed (~17k lines): the FastAPI app (web_api, schemas, health_check,
metrics, postgres_history, forecast_history, thaiwater), the Python RID
collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix
alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL,
InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/
logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling
(api_parity, collector_parity, e2e_notify, install_collector.sh,
gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files,
sql/, the release workflow and the docs for all of that. Their tests went
with them.
Kept in Python: src/ml (training, evaluation harness, export; predict.py as
the reference the Rust forecaster is tested against), the HII and RID-dam
clients/stores for the archive backfills (their live collectors removed),
the backfill scripts, retrain.sh + timer, forecast_parity.py.
- PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env;
the stores lost their sqlite/mysql branches (and the 2026-08 column
migration). The database tests run against a throwaway schema from
TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the
Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB).
- The notifier settings moved from src/notify.py into src/ml/export.py;
flood_config.json regenerates byte-identical.
- tests/test_dashboard.py checks every path the dashboard fetches against
ping-collector's router.
- Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil,
schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv
1.1.1 explicit (it came in through uvicorn[standard]).
- Formatting contract extended to scripts/ and tests/ (reformatted once).
- scripts/install.sh is the one installer (fresh host or update, in place in
a checkout without touching its ownership): binary, units, uv sync, .env
defaults, removes the retired water-monitor.service and any old
--alert-check cron job, dry run + forecast check, restart, readiness wait.
- Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments
and Rust doc comments updated to the Rust service.
security.yml previously ran safety/bandit/semgrep with `|| true` and could
not go red. Now: pip-audit on requirements.txt is a hard gate (dev deps
reported only), bandit HIGH fails (B104 bind-all skipped: intended behind
Cloudflare/Caddy), pip-licenses uploaded as a report. Weekly + on
dependency/source changes.
Running it locally found 29 advisories, all in pinned-and-forgotten
runtime deps: starlette 0.27 (7, incl. Host-header path confusion and
form DoS), fastapi 0.104, requests 2.31 (3), pymysql 1.1. Bumped to
current: fastapi 0.141.1 / starlette 1.6.0, pydantic 2.13.5, uvicorn
0.52.4, requests 2.34.2, pymysql 1.2.0; dev: pytest 9.1.1, black 26.5.1.
pip-audit is now clean. requires-python narrowed to 3.11 (the truth:
psycopg2-binary 2.9.9 fails on 3.13; pandas 2.0.3 has no 3.12 wheels).
Full suite passes; API smoke-tested (health, stations, forecast, history,
stats, docs, openapi) on the new stack. black 26 reformatted 8 files.
scikit-learn 1.9.0 supports only Python >=3.11, so uv could not resolve
the >=3.9 range. The deployment runs 3.11. Also pins numpy back to
1.26.4 in the lockfile (pandas 2.0.3 ABI).
- **Migration to uv package manager**: Replace pip/requirements with modern pyproject.toml
- Add pyproject.toml with complete dependency management
- Update all scripts and Makefile to use uv commands
- Maintain backward compatibility with existing workflows
- **PostgreSQL integration and migration tools**:
- Enhanced config.py with automatic password URL encoding
- Complete PostgreSQL setup scripts and documentation
- High-performance SQLite to PostgreSQL migration tool (91x speed improvement)
- Support for both connection strings and individual components
- **Executable distribution system**:
- PyInstaller integration for standalone .exe creation
- Automated build scripts with batch file generation
- Complete packaging system for end-user distribution
- **Enhanced data management**:
- Fix --fill-gaps command with proper method implementation
- Add gap detection and historical data backfill capabilities
- Implement data update functionality for existing records
- Add comprehensive database adapter methods
- **Developer experience improvements**:
- Password encoding tools for special characters
- Interactive setup wizards for PostgreSQL configuration
- Comprehensive documentation and migration guides
- Automated testing and validation tools
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
grabowski
