feat(collector): forecast and ntfy in ping-collector; ThaiWater keeps alerts current at night
CI / Format & lint (push) Successful in 22s
CI / Test suite (push) Successful in 59s
Docs / Validate documentation (push) Successful in 25s
Rust / fmt, clippy, test, build (push) Successful in 10m2s
Security / Dependency vulnerabilities (push) Successful in 2m11s
Security / Static analysis (push) Successful in 22s
Security / License report (push) Successful in 1m37s

Phase 2b/2c of docs/RUST_REWRITE.md. With FORECAST_IN_COLLECTOR=true (shared
.env) ping-collector, after each cycle that brought newer RID or HII readings
and at least hourly: loads the last 336 h (RID + HII mirror fill), fetches the
rain forecast, runs every exported model (rust/forecast), stores
forecast_history and forecast_current, and sends the ntfy notifications.
The Python API then serves forecast_current as /forecast (computing in-process
only when it is older than FORECAST_MAX_AGE_SECONDS) and neither forecasts nor
notifies.

ntfy (collector/src/notify.rs): a port of src/notify.py with the same topics,
texts, hysteresis, capacity guard and notification_state keys, checked word
for word on 751 messages generated by Python (testdata/gen_notify_cases.py).
The change: for stations whose ThaiWater twin is the same sensor, a reading
newer than RID's drives the level alerts (marked as ThaiWater telemetry), and
the feed counts as stale only when neither source has a reading for 3 h. RID
alone lagging is reported once after RID_STALE_ALERT_HOURS (18). This ends the
nightly "gauge feed stale" / "recovered" pairs caused by RID's late reports.

Commands: `forecast [--json] [--save] [--now] [--rain-file]`,
`notify --dry-run`. scripts/forecast_parity.py compares Python and the
collector on the live database (same moment, same rain); locally 48/48 rows
identical. install_collector.sh: models export + that check on every install,
--cutover-forecast (refuses on a failed check) and --rollback-forecast.

Trees are packed into 24-byte nodes: 70 MB peak with all 15 models (was 140).
Forecast values are written to NUMERIC as shortest round-trip text, so they
round as psycopg2's do. DB test schemas get a counter (parallel tests collided
on Windows' coarse clock).
This commit is contained in:
grabowski committed 2026-10-18 14:43:39 +07:00
1 parent 5c4f464383
commit 6ae67dec63
27 files changed
+2656 -118

No files matched your search

+299
View File
@@ -131,6 +131,31 @@ const SCHEMA: &[&str] = &[
mae_rim NUMERIC(6,2), chiang_mai NUMERIC(6,2),
catchment_mean NUMERIC(6,2),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)",
// src/forecast_history.py
"CREATE TABLE IF NOT EXISTS forecast_history (
as_of TIMESTAMP NOT NULL,
station_code VARCHAR(10) NOT NULL,
horizon_hours INTEGER NOT NULL,
predicted_max_level NUMERIC(8,3),
p_warning NUMERIC(7,5),
p_danger NUMERIC(7,5),
current_level NUMERIC(8,3),
model_version VARCHAR(64),
source VARCHAR(16),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (as_of, station_code, horizon_hours))",
"CREATE INDEX IF NOT EXISTS idx_forecast_history_station ON forecast_history(station_code, as_of)",
// src/notify.py
"CREATE TABLE IF NOT EXISTS notification_state (
key VARCHAR(64) PRIMARY KEY, state VARCHAR(16) NOT NULL,
value NUMERIC(8,3), updated_at TIMESTAMP NOT NULL)",
// The current /forecast payload, read by the Python API when
// FORECAST_IN_COLLECTOR=true (one row, id 'all').
"CREATE TABLE IF NOT EXISTS forecast_current (
id VARCHAR(16) PRIMARY KEY,
computed_at TIMESTAMP NOT NULL,
model_versions TEXT,
payload TEXT NOT NULL)",
];
pub async fn ensure_schema(pool: &PgPool) -> Result<()> {
@@ -481,3 +506,277 @@ pub async fn table_counts(pool: &PgPool) -> Result<BTreeMap<&'static str, i64>>
}
Ok(out)
}
// ---------------------------------------------------------------------------
// Forecast inputs and outputs (FORECAST_IN_COLLECTOR)
//
// NUMERIC columns are read as text and parsed: Python's `float(Decimal)` is a
// correctly rounded parse of the same digits, so both sides get the same f64.
// Forecast values are written as text (shortest round-trip digits, as Python's
// repr) so PostgreSQL rounds the same decimal as it does for psycopg2.
// ---------------------------------------------------------------------------
/// A row of (key, key, NUMERIC as text, NUMERIC as text).
type TextRow<A, B> = (A, B, Option<String>, Option<String>);
/// (date, storage_pct, inflow_mcm, outflow_mcm) with NUMERIC as text.
type DamRow = (NaiveDate, Option<String>, Option<String>, Option<String>);
fn parse_num(v: Option<String>) -> f64 {
v.and_then(|s| s.parse().ok()).unwrap_or(f64::NAN)
}
fn num_text(v: f64) -> Option<String> {
v.is_finite().then(|| format!("{v}"))
}
/// RID readings in [start, end], as `data._fetch_from_db`.
pub async fn load_rid_window(
pool: &PgPool,
start: NaiveDateTime,
end: NaiveDateTime,
) -> Result<Vec<ping_forecast::Row>> {
let rows: Vec<TextRow<NaiveDateTime, String>> = sqlx::query_as(
"SELECT m.timestamp, s.station_code, m.water_level::text, m.discharge::text
FROM water_measurements m JOIN stations s ON m.station_id = s.id
WHERE m.timestamp >= $1 AND m.timestamp <= $2 ORDER BY m.timestamp",
)
.bind(start)
.bind(end)
.fetch_all(pool)
.await
.context("loading the RID window")?;
Ok(rows
.into_iter()
.map(|(t, code, level, q)| ping_forecast::Row {
timestamp: ping_forecast::grid::floor_hour(t),
station_code: code,
water_level: parse_num(level),
discharge: parse_num(q),
})
.collect())
}
/// HII mirror readings for `codes` in [start, end], as `data._fetch_hii_levels`.
pub async fn load_hii_window(
pool: &PgPool,
codes: &[String],
start: NaiveDateTime,
end: NaiveDateTime,
) -> Result<Vec<ping_forecast::hii::HiiRow>> {
let rows: Vec<TextRow<NaiveDateTime, String>> = sqlx::query_as(
"SELECT m.timestamp, s.rid_code, m.wl_msl::text, m.discharge::text
FROM hii_waterlevel m JOIN hii_wl_stations s ON s.id = m.station_id
WHERE s.rid_code IS NOT NULL AND m.timestamp >= $1 AND m.timestamp <= $2
AND s.rid_code = ANY($3)
ORDER BY m.timestamp, m.station_id",
)
.bind(start)
.bind(end)
.bind(codes)
.fetch_all(pool)
.await
.context("loading HII mirror levels")?;
Ok(rows
.into_iter()
.map(|(t, code, wl, q)| ping_forecast::hii::HiiRow {
timestamp: t,
station_code: code,
wl_msl: parse_num(wl),
discharge: parse_num(q),
})
.collect())
}
/// Newest HII mirror reading for `codes` (a forecast trigger).
pub async fn latest_hii_timestamp(
pool: &PgPool,
codes: &[String],
) -> Result<Option<NaiveDateTime>> {
Ok(sqlx::query_scalar(
"SELECT MAX(m.timestamp) FROM hii_waterlevel m JOIN hii_wl_stations s ON s.id = m.station_id
WHERE s.rid_code = ANY($1) AND m.wl_msl IS NOT NULL",
)
.bind(codes)
.fetch_one(pool)
.await?)
}
/// Daily reservoir rows from `start`, as `dam.load_daily`.
pub async fn load_dam_days(
pool: &PgPool,
dam_id: &str,
start: NaiveDate,
) -> Result<Vec<ping_forecast::dam::DamDay>> {
let rows: Vec<DamRow> = sqlx::query_as(
"SELECT date, storage_pct::text, inflow_mcm::text, outflow_mcm::text
FROM rid_reservoir_daily WHERE dam_id = $1 AND date >= $2 ORDER BY date",
)
.bind(dam_id)
.bind(start)
.fetch_all(pool)
.await
.context("loading the dam series")?;
Ok(rows
.into_iter()
.map(|(date, s, i, o)| ping_forecast::dam::DamDay {
date,
storage_pct: parse_num(s),
inflow_mcm: parse_num(i),
outflow_mcm: parse_num(o),
})
.collect())
}
/// The newest RID row per station (`get_latest_measurements(200)`).
#[derive(Clone, Debug, PartialEq)]
pub struct LatestReading {
pub timestamp: NaiveDateTime,
pub station_code: String,
pub water_level: Option<f64>,
pub discharge_percent: Option<f64>,
}
pub async fn latest_rid_readings(pool: &PgPool) -> Result<Vec<LatestReading>> {
let rows: Vec<TextRow<NaiveDateTime, String>> = sqlx::query_as(
"SELECT m.timestamp, s.station_code, m.water_level::text, m.discharge_percent::text
FROM water_measurements m
JOIN stations s ON m.station_id = s.id
INNER JOIN (
SELECT station_id, MAX(timestamp) AS max_timestamp
FROM water_measurements GROUP BY station_id
) latest ON m.station_id = latest.station_id AND m.timestamp = latest.max_timestamp
ORDER BY s.station_code
LIMIT 200",
)
.fetch_all(pool)
.await
.context("loading the latest readings")?;
let num = |v: Option<String>| v.and_then(|s| s.parse().ok());
Ok(rows
.into_iter()
.map(|(t, code, level, pct)| LatestReading {
timestamp: t,
station_code: code,
water_level: num(level),
discharge_percent: num(pct),
})
.collect())
}
/// `ForecastHistoryStore.save_rows`: one row per (as_of, station, horizon).
pub async fn save_forecast_history(
pool: &PgPool,
rows: &[ping_forecast::Forecast],
) -> Result<usize> {
let rows = dedupe_last(rows, |r| (r.as_of, r.station_code.clone(), r.horizon_hours));
if rows.is_empty() {
return Ok(0);
}
sqlx::query(
"INSERT INTO forecast_history (as_of, station_code, horizon_hours, predicted_max_level,
p_warning, p_danger, current_level, model_version, source)
SELECT a, c, h, pm::numeric, pw::numeric, pd::numeric, cl::numeric, mv, src
FROM UNNEST($1::timestamp[], $2::text[], $3::int4[], $4::text[], $5::text[], $6::text[],
$7::text[], $8::text[], $9::text[])
AS t(a, c, h, pm, pw, pd, cl, mv, src)
ON CONFLICT (as_of, station_code, horizon_hours) DO UPDATE SET
predicted_max_level = EXCLUDED.predicted_max_level, p_warning = EXCLUDED.p_warning,
p_danger = EXCLUDED.p_danger, current_level = EXCLUDED.current_level,
model_version = EXCLUDED.model_version, source = EXCLUDED.source",
)
.bind(rows.iter().map(|r| r.as_of).collect::<Vec<_>>())
.bind(
rows.iter()
.map(|r| r.station_code.clone())
.collect::<Vec<_>>(),
)
.bind(
rows.iter()
.map(|r| r.horizon_hours as i32)
.collect::<Vec<_>>(),
)
.bind(
rows.iter()
.map(|r| num_text(r.predicted_max_level))
.collect::<Vec<_>>(),
)
.bind(
rows.iter()
.map(|r| num_text(r.p_warning))
.collect::<Vec<_>>(),
)
.bind(
rows.iter()
.map(|r| num_text(r.p_danger))
.collect::<Vec<_>>(),
)
.bind(
rows.iter()
.map(|r| num_text(r.current_level))
.collect::<Vec<_>>(),
)
.bind(
rows.iter()
.map(|r| r.model_version.clone())
.collect::<Vec<_>>(),
)
.bind(rows.iter().map(|r| r.source.clone()).collect::<Vec<_>>())
.execute(pool)
.await
.context("upserting forecast_history")?;
Ok(rows.len())
}
/// The payload the Python API serves as /forecast.
pub async fn save_forecast_current(
pool: &PgPool,
computed_at: NaiveDateTime,
rows: &[ping_forecast::Forecast],
) -> Result<()> {
let versions: std::collections::BTreeSet<&str> =
rows.iter().map(|r| r.model_version.as_str()).collect();
sqlx::query(
"INSERT INTO forecast_current (id, computed_at, model_versions, payload)
VALUES ('all', $1, $2, $3)
ON CONFLICT (id) DO UPDATE SET computed_at = EXCLUDED.computed_at,
model_versions = EXCLUDED.model_versions, payload = EXCLUDED.payload",
)
.bind(computed_at)
.bind(versions.into_iter().collect::<Vec<_>>().join(","))
.bind(serde_json::to_string(rows)?)
.execute(pool)
.await
.context("writing forecast_current")?;
Ok(())
}
/// notification_state as a key -> state map.
pub async fn notification_states(pool: &PgPool) -> Result<HashMap<String, String>> {
let rows: Vec<(String, String)> = sqlx::query_as("SELECT key, state FROM notification_state")
.fetch_all(pool)
.await
.context("reading notification_state")?;
Ok(rows.into_iter().collect())
}
pub async fn set_notification_state(
pool: &PgPool,
key: &str,
state: &str,
value: Option<f64>,
now: NaiveDateTime,
) -> Result<()> {
sqlx::query(
"INSERT INTO notification_state (key, state, value, updated_at)
VALUES ($1, $2, $3::numeric, $4) ON CONFLICT (key) DO UPDATE SET
state = EXCLUDED.state, value = EXCLUDED.value, updated_at = EXCLUDED.updated_at",
)
.bind(key)
.bind(state)
.bind(value.and_then(num_text))
.bind(now)
.execute(pool)
.await
.with_context(|| format!("saving notification state {key}"))?;
Ok(())
}