rust/collector replaces the collection half of the Python web app: RID hourly
gauges (with the late-report semantics: store everything, retry every 10 min
while RID is behind, re-fetch yesterday's holes, startup gap fill), HII rain +
water level, RID dams (today + yesterday) and Open-Meteo rain. Same tables and
upsert semantics, one UNNEST statement per table, Bangkok wall-clock time
regardless of the server time zone.
Python side: COLLECTION_ENABLED=false makes the API leader stop collecting and
instead re-forecast + notify when the newest RID reading in the database
advances (polled every FORECAST_POLL_SECONDS) and once per clock hour;
/scraping/status then serves the collector's status file; /scrape/trigger
answers 409.
Verified: 20 Rust tests (parsers, gap/schedule logic, database tests against
PostgreSQL); live parity run of both collectors into two empty databases:
all 9 tables row-for-row identical (scripts/collector_parity.py); the Python
API served health, measurements, stats, status and model forecasts from a
database filled only by ping-collector. 177 Python tests pass.
reqwest uses native-tls: app.rid.go.th offers only TLS 1.2 CBC suites, which
rustls refuses.
Deploy/cutover/rollback: scripts/install_collector.sh, scripts/ping-collector.service;
CI: .gitea/workflows/rust.yml (fmt, clippy, tests with a PostgreSQL service,
release artifact).
RID's hourly report often lags 1-2 h in the evening. The cycle discarded the
whole report unless it already held the current hour, the web-API loop then
slept a full hour, and after 01:00 only today's report was fetched. Result:
the 19:00-24:00 readings were dropped most nights (P.1 lost 00:00 on 23 of
the last 60 days, 13 evening hours this week), the feed went stale from
~21:00 until morning and ntfy ping-status sent a stale/recovered pair daily.
- run_scraping_cycle() upserts every reading the report has; "current hour
present" now only decides the retry cadence (returned as before).
- Yesterday's report is re-fetched while the store has holes in it (hours
22-24 are only reachable through that day's report).
- Leader loop polls at :05 (RID_POLL_OFFSET_MINUTES) and retries every
RID_RETRY_MINUTES (10) while RID is behind; HII/dams/rain once per clock
hour; forecast precompute + ntfy when newer readings arrive or hourly.
- Startup re-fetches holes from the last GAP_FILL_DAYS_ON_START (14) days.
- /scraping/status: the leader shares its stats via a JSON file, so any
worker reports them (was always 0 runs); adds feed_current,
latest_reading, readings_saved.
The reverse proxy is a separate VPS on the tailnet, so a loopback-only
ntfy was unreachable from it. install_ntfy.sh now binds the host's Tailscale
IP (NTFY_LISTEN overrides). New NTFY_PUBLISH_URL: where the monitor POSTs,
separate from the public NTFY_SERVER subscribers see, so an alert never
waits on DNS or the proxy (first cycle logged 502s from Cloudflare while
the domain was not yet proxied).
Anyone can now get push alerts on their phone without an account: the
monitor publishes to an ntfy server (one Go binary, ~30 MB RSS) and
subscribers pick topics in the free iOS/Android/web app.
Semantics are transitions, never state. One message when a gauge crosses
its warning or danger threshold, one all-clear when it drops back (0.10 m
hysteresis), nothing while it sits above. A three-day flood is two
messages; a quiet season is zero. Topics: ping-warning / ping-danger
(basin digest), ping-<station>-warning / -danger, ping-p1-outlook (opt-in:
model P(warning within 24 h) at P.1 rises through 50 %, clears below 25 %,
message says it is experimental), ping-status (feed stale >= 3 h /
recovered). Priority 5 on danger so it rings through Do Not Disturb.
src/notify.py runs once per collection cycle in the API process (leader
only, after the forecast precompute, same data the dashboard shows).
Last-sent state lives in a notification_state table so a restart never
re-sends; a failed publish leaves state untouched so the crossing is
retried next cycle instead of lost. Off unless NTFY_SERVER is set.
Dashboard: a "Get alerts" button (only when configured) opens a panel
with the server, per-topic cards, ntfy:// deep links and web links, app
store links and a disclaimer. EN + TH. GET /api/notifications feeds it.
scripts/install_ntfy.sh: .deb install, server.yml (loopback listen,
anonymous read, token-only write scoped to ping-*, 72 h cache, signup/
login/metrics off, tight visitor limits), systemd, user + token, .env.
Verified against ntfy 2.28.0: anon publish 403, token publish 200, token
on foreign topic 403, anon read 200, and a seeded crossing through the
real _notify_transitions path arrived in the topic with priority, tags,
click and action button. docs/NOTIFICATIONS.md has the deployment and
reverse-proxy notes. Tests: 10 for the state machine (159 total).
- Implement custom Python alerting system (src/alerting.py) with water level monitoring, data freshness checks, and Matrix notifications
- Add complete Grafana Matrix alerting setup guide (docs/GRAFANA_MATRIX_SETUP.md) with webhook configuration, alert rules, and notification policies
- Create Matrix quick start guide (docs/MATRIX_QUICK_START.md) for rapid deployment
- Integrate alerting commands into main application (--alert-check, --alert-test)
- Add Matrix configuration to environment variables (.env.example)
- Update Makefile with alerting targets (alert-check, alert-test)
- Enhance status command to show Matrix notification status
- Support station-specific water level thresholds and escalation rules
- Provide dual alerting approach: native Grafana alerts and custom Python system
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- **Migration to uv package manager**: Replace pip/requirements with modern pyproject.toml
- Add pyproject.toml with complete dependency management
- Update all scripts and Makefile to use uv commands
- Maintain backward compatibility with existing workflows
- **PostgreSQL integration and migration tools**:
- Enhanced config.py with automatic password URL encoding
- Complete PostgreSQL setup scripts and documentation
- High-performance SQLite to PostgreSQL migration tool (91x speed improvement)
- Support for both connection strings and individual components
- **Executable distribution system**:
- PyInstaller integration for standalone .exe creation
- Automated build scripts with batch file generation
- Complete packaging system for end-user distribution
- **Enhanced data management**:
- Fix --fill-gaps command with proper method implementation
- Add gap detection and historical data backfill capabilities
- Implement data update functionality for existing records
- Add comprehensive database adapter methods
- **Developer experience improvements**:
- Password encoding tools for special characters
- Interactive setup wizards for PostgreSQL configuration
- Comprehensive documentation and migration guides
- Automated testing and validation tools
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Features:
- Real-time water level monitoring for Ping River Basin (16 stations)
- Coverage from Chiang Dao to Nakhon Sawan in Northern Thailand
- FastAPI web interface with interactive dashboard and station management
- Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics)
- Comprehensive monitoring with health checks and metrics collection
- Docker deployment with Grafana integration
- Production-ready architecture with enterprise-grade observability
CI/CD & Automation:
- Complete Gitea Actions workflows for CI/CD, security, and releases
- Multi-Python version testing (3.9-3.12)
- Multi-architecture Docker builds (amd64, arm64)
- Daily security scanning and dependency monitoring
- Automated documentation generation
- Performance testing and validation
Production Ready:
- Type safety with Pydantic models and comprehensive type hints
- Data validation layer with range checking and error handling
- Rate limiting and request tracking for API protection
- Enhanced logging with rotation, colors, and performance metrics
- Station management API for dynamic CRUD operations
- Comprehensive documentation and deployment guides
Technical Stack:
- Python 3.9+ with FastAPI and Pydantic
- Multi-database architecture with adapter pattern
- Docker containerization with multi-stage builds
- Grafana dashboards for visualization
- Gitea Actions for CI/CD automation
- Enterprise monitoring and alerting
Ready for deployment to B4L infrastructure!
grabowski
