grabowski
779728e38b
Phase 3 of docs/RUST_REWRITE.md: rust/collector/src/web (axum) serves what
the FastAPI app (src/web_api.py) served, so one process collects, forecasts,
notifies and serves. With WEB_LISTEN=0.0.0.0:8000 it runs inside
`ping-collector run`; `ping-collector serve` runs it alone.
Same routes, parameters, JSON fields and field order, status codes,
{"detail": ...} errors and pydantic-style 422 lists, Cache-Control, gzip
(>500 B), CORS, Umami API tracking, the dashboard and static files, and the
stale-while-revalidate response caches (now with byte budgets: a history
response can be 10 MB). /docs is the same Swagger UI (+ /redoc); its OpenAPI
document is built from the route table, whose per-route parameter
declarations also validate the requests, so docs and behaviour cannot drift.
Python output conventions are kept: isoformat(), round(), numpy's pairwise
sums (catchment means), Python's summation order (skill metrics).
scripts/api_parity.py compares the two servers route by route on one
database (53 cases incl. errors, admin and static files; query values vary
per run so FastAPI's caches answer fresh). Locally: 53 identical; the
station CRUD writes byte-identical stations.json files; the skill/trend
logic matches on 9 station/horizon pairs with synthetic model versions.
Intended differences: /health counts the ThaiWater telemetry of the same
gauges while RID lags (as the notifier); /scraping/status adds the collector
version and last forecast run; an old stored forecast is served marked
X-Data-Stale instead of recomputed; /sensors/thaiwater is cached a minute;
.geojson is served as application/geo+json. Load (local, 16 clients): the
dashboard's routes 1375 vs 552 req/s, uncached history 714 vs 53 req/s;
25 MB resident when serving only.
Matrix (MATRIX_*): the notifier sends the room the transitions ntfy gets
(per station, outlook, feed; not ntfy's per-topic summaries), each once,
with its own matrix: keys in notification_state, seeded from ntfy's on first
use so nothing already announced repeats. Checked against a fake homeserver.
It replaces the `run.py --alert-check` cron job (repeated alerts every 15
min), which install_collector.sh now comments out (crontab backup kept).
install_collector.sh: --check-api (ping-collector serve on :8001 next to
FastAPI, api_parity.py), --cutover-api (refuses on a difference; sets
WEB_LISTEN, stops and disables water-monitor.service), --rollback-api;
--rollback-forecast and --rollback bring the Python app back as server.
ping-collector.service: MemoryMax 512M. Config::from_vars for tests; route
tests against a throwaway schema.
141 lines
4.5 KiB
Bash
141 lines
4.5 KiB
Bash
# Northern Thailand Ping River Monitor Configuration
|
|
# Copy this file to .env and customize for your environment
|
|
|
|
# Database Configuration
|
|
DB_TYPE=postgresql
|
|
# Options: sqlite, mysql, postgresql, influxdb, victoriametrics
|
|
|
|
# SQLite Configuration (default)
|
|
WATER_DB_PATH=water_levels.db
|
|
|
|
# VictoriaMetrics Configuration
|
|
VM_HOST=localhost
|
|
VM_PORT=8428
|
|
VM_URL=
|
|
|
|
# InfluxDB Configuration
|
|
INFLUX_HOST=localhost
|
|
INFLUX_PORT=8086
|
|
INFLUX_DATABASE=ping_river_monitoring
|
|
INFLUX_USERNAME=
|
|
INFLUX_PASSWORD=
|
|
|
|
# PostgreSQL Configuration (Remote Server)
|
|
# Option 1: Full connection string (URL encode special characters in password)
|
|
POSTGRES_CONNECTION_STRING=postgresql://username:url_encoded_password@your-postgres-host:5432/water_monitoring
|
|
|
|
# Option 2: Individual components (password will be automatically URL encoded)
|
|
POSTGRES_HOST=your-postgres-host
|
|
POSTGRES_PORT=5432
|
|
POSTGRES_DB=water_monitoring
|
|
POSTGRES_USER=username
|
|
POSTGRES_PASSWORD=your:password@with!special#chars
|
|
|
|
# Examples for connection string:
|
|
# - Local: postgresql://postgres:password@localhost:5432/water_monitoring
|
|
# - Remote: postgresql://user:pass@192.168.1.100:5432/water_monitoring
|
|
# - With special chars: postgresql://user:my%3Apass%40word@host:5432/db
|
|
# - With SSL: postgresql://user:pass@host:port/db?sslmode=require
|
|
# - Connection pooling: postgresql://user:pass@host:port/db?pool_size=20&max_overflow=0
|
|
|
|
# Special character URL encoding:
|
|
# : → %3A @ → %40 # → %23 ? → %3F & → %26 / → %2F % → %25
|
|
|
|
# MySQL Configuration
|
|
MYSQL_CONNECTION_STRING=mysql://user:password@localhost:3306/ping_river_monitoring
|
|
|
|
# API Configuration
|
|
API_HOST=0.0.0.0
|
|
API_PORT=8000
|
|
API_WORKERS=1
|
|
# Public ThaiWater API key used to add Ping-basin water-level sensors.
|
|
THAIWATER_API_KEY=
|
|
|
|
# Data Collection Settings
|
|
SCRAPING_INTERVAL_HOURS=1
|
|
# RID poll minute past the hour; retry interval while RID is behind
|
|
RID_POLL_OFFSET_MINUTES=5
|
|
RID_RETRY_MINUTES=10
|
|
# Days of RID holes the leader re-fetches at startup (0 = off)
|
|
GAP_FILL_DAYS_ON_START=14
|
|
# false when the Rust ping-collector service collects (rust/collector);
|
|
# share the status file between both services for /scraping/status
|
|
COLLECTION_ENABLED=true
|
|
# SCRAPING_STATUS_FILE=/opt/thailand-water-monitor/collector-status.json
|
|
# true: ping-collector also forecasts (models/flood_*.json.gz, rust/forecast)
|
|
# and sends the ntfy notifications; this app serves its stored forecast and
|
|
# computes itself only when that is older than FORECAST_MAX_AGE_SECONDS.
|
|
# Set by scripts/install_collector.sh --cutover-forecast.
|
|
FORECAST_IN_COLLECTOR=false
|
|
# FORECAST_MAX_AGE_SECONDS=7200
|
|
# MODELS_DIR=models
|
|
REQUEST_TIMEOUT=30
|
|
MAX_RETRIES=3
|
|
RETRY_DELAY_SECONDS=60
|
|
|
|
# Data Retention
|
|
DATA_RETENTION_DAYS=365
|
|
|
|
# Logging Configuration
|
|
LOG_LEVEL=INFO
|
|
LOG_FILE=water_monitor.log
|
|
|
|
# Security (for production)
|
|
SECRET_KEY=your-secret-key-here
|
|
API_KEY=your-api-key-here
|
|
|
|
# Monitoring
|
|
ENABLE_METRICS=true
|
|
ENABLE_HEALTH_CHECKS=true
|
|
|
|
# Geographic Settings
|
|
TIMEZONE=Asia/Bangkok
|
|
DEFAULT_LATITUDE=18.7875
|
|
DEFAULT_LONGITUDE=99.0045
|
|
|
|
# External Services
|
|
NOTIFICATION_EMAIL=
|
|
SMTP_SERVER=
|
|
SMTP_PORT=587
|
|
SMTP_USERNAME=
|
|
SMTP_PASSWORD=
|
|
|
|
# Public push notifications via self-hosted ntfy (https://ntfy.sh, single binary).
|
|
# Leave NTFY_SERVER empty to disable. Topics published: <prefix>-<station>-warning,
|
|
# <prefix>-<station>-danger, <prefix>-warning, <prefix>-danger, <prefix>-p1-outlook,
|
|
# <prefix>-status. See docs/NOTIFICATIONS.md.
|
|
NTFY_SERVER=
|
|
# Where the monitor POSTs (defaults to NTFY_SERVER). Use the local ntfy
|
|
# address (loopback or Tailscale IP) so publishing does not depend on
|
|
# DNS / the reverse proxy being up.
|
|
NTFY_PUBLISH_URL=
|
|
NTFY_TOPIC_PREFIX=ping
|
|
NTFY_TOKEN=
|
|
PUBLIC_URL=https://water.buildfor.life/
|
|
# ping-collector notifier: report RID lagging this many hours while ThaiWater
|
|
# telemetry is current (0 = never)
|
|
RID_STALE_ALERT_HOURS=18
|
|
|
|
# Matrix: ping-collector sends the ntfy transitions to this room too
|
|
# (docs/MATRIX_QUICK_START.md); off while token or room is empty
|
|
MATRIX_HOMESERVER=https://matrix.org
|
|
MATRIX_ACCESS_TOKEN=
|
|
MATRIX_ROOM_ID=
|
|
|
|
# Website and API served by ping-collector (rust/collector/src/web): the
|
|
# address to listen on; empty = the Python app serves (port 8000). Set by
|
|
# scripts/install_collector.sh --cutover-api.
|
|
WEB_LISTEN=
|
|
# STATIC_DIR=src/static
|
|
# ADMIN_API_KEY= # X-API-Key for station editing, /config, /scrape/trigger
|
|
|
|
# Grafana Integration
|
|
GRAFANA_URL=http://localhost:3000
|
|
|
|
# Alert Configuration
|
|
ALERT_MAX_AGE_HOURS=2
|
|
ALERT_CHECK_INTERVAL_MINUTES=15
|
|
|
|
# Development Settings
|
|
DEBUG=false
|
|
DEVELOPMENT_MODE=false |