feat(collector): forecast and ntfy in ping-collector; ThaiWater keeps alerts current at night
CI / Format & lint (push) Successful in 22s
CI / Test suite (push) Successful in 59s
Docs / Validate documentation (push) Successful in 25s
Rust / fmt, clippy, test, build (push) Successful in 10m2s
Security / Dependency vulnerabilities (push) Successful in 2m11s
Security / Static analysis (push) Successful in 22s
Security / License report (push) Successful in 1m37s

Phase 2b/2c of docs/RUST_REWRITE.md. With FORECAST_IN_COLLECTOR=true (shared
.env) ping-collector, after each cycle that brought newer RID or HII readings
and at least hourly: loads the last 336 h (RID + HII mirror fill), fetches the
rain forecast, runs every exported model (rust/forecast), stores
forecast_history and forecast_current, and sends the ntfy notifications.
The Python API then serves forecast_current as /forecast (computing in-process
only when it is older than FORECAST_MAX_AGE_SECONDS) and neither forecasts nor
notifies.

ntfy (collector/src/notify.rs): a port of src/notify.py with the same topics,
texts, hysteresis, capacity guard and notification_state keys, checked word
for word on 751 messages generated by Python (testdata/gen_notify_cases.py).
The change: for stations whose ThaiWater twin is the same sensor, a reading
newer than RID's drives the level alerts (marked as ThaiWater telemetry), and
the feed counts as stale only when neither source has a reading for 3 h. RID
alone lagging is reported once after RID_STALE_ALERT_HOURS (18). This ends the
nightly "gauge feed stale" / "recovered" pairs caused by RID's late reports.

Commands: `forecast [--json] [--save] [--now] [--rain-file]`,
`notify --dry-run`. scripts/forecast_parity.py compares Python and the
collector on the live database (same moment, same rain); locally 48/48 rows
identical. install_collector.sh: models export + that check on every install,
--cutover-forecast (refuses on a failed check) and --rollback-forecast.

Trees are packed into 24-byte nodes: 70 MB peak with all 15 models (was 140).
Forecast values are written to NUMERIC as shortest round-trip text, so they
round as psycopg2's do. DB test schemas get a counter (parallel tests collided
on Windows' coarse clock).
This commit is contained in:
grabowski committed 2026-10-18 14:43:39 +07:00
1 parent 5c4f464383
commit 6ae67dec63
27 files changed
+2656 -118

No files matched your search

+22
View File
@@ -35,6 +35,19 @@ pub struct Config {
pub station_config_path: PathBuf,
/// Status JSON read by the Python API's /scraping/status.
pub status_file: Option<PathBuf>,
/// FORECAST_IN_COLLECTOR: forecast and notify here; the Python API then
/// serves the stored forecast and does neither.
pub forecast_enabled: bool,
/// Exported bundles (`flood_<station>.json.gz`, src/ml/export.py).
pub models_dir: PathBuf,
/// Where to POST ntfy messages; `None` (NTFY_SERVER unset) = off.
pub ntfy_publish_url: Option<String>,
pub ntfy_token: Option<String>,
pub ntfy_topic_prefix: String,
/// Dashboard link in notifications.
pub public_url: String,
/// Report RID lagging this long while ThaiWater is current (0 = never).
pub rid_stale_alert_hours: f64,
}
fn var(name: &str) -> Option<String> {
@@ -125,6 +138,15 @@ impl Config {
.unwrap_or_else(|| "stations.json".into())
.into(),
status_file: var("SCRAPING_STATUS_FILE").map(PathBuf::from),
forecast_enabled: flag("FORECAST_IN_COLLECTOR", false),
models_dir: var("MODELS_DIR").unwrap_or_else(|| "models".into()).into(),
// as src/config.py: off unless NTFY_SERVER is set
ntfy_publish_url: var("NTFY_SERVER")
.map(|server| var("NTFY_PUBLISH_URL").unwrap_or(server)),
ntfy_token: var("NTFY_TOKEN"),
ntfy_topic_prefix: var("NTFY_TOPIC_PREFIX").unwrap_or_else(|| "ping".into()),
public_url: var("PUBLIC_URL").unwrap_or_else(|| "https://water.buildfor.life/".into()),
rid_stale_alert_hours: number("RID_STALE_ALERT_HOURS", 18.0)?,
};
if cfg.poll_offset_minutes >= 60 {
bail!("RID_POLL_OFFSET_MINUTES must be between 0 and 59");