feat(rid): screen one-hour spikes; refetch and redo-from to repair the store
RID's report sometimes carries a wrong value for one hour and corrects it later. P.1 read 10.6 m at 2026-10-25 00:00 between 1.04 and 1.07 (RID now says 1.06); the store kept it, because a report is re-fetched only while it has holes, and the model issued a 10.8 m forecast for that hour. - rid::judge / rid::screen: a reading that leaves the previous one (within 3 h) by more than 1 m (2 m on the flashy tributaries P.76, P.77, P.81, P.82, P.84, P.85, P.87) while the next reading comes back near the previous level is a spike: not stored, a stored copy is deleted. Limbs and steps the next reading confirms are kept; the newest reading waits for the next one when it jumps more than twice the limit. On 2018-2026 history: 16 spikes found (slipped decimals, sign flips, -5/-6/-9 m placeholders), no genuine change rejected. - Every store goes through it (rid::store: run_cycle, fill-gaps, refetch). - `ping-collector refetch FROM [TO]`: re-fetch reports regardless of holes (corrects revised readings, fills the discharge of level-only evening hours, removes stored spikes) and print the catch-up command when readings changed. - `catch-up-forecasts --redo-from T`: delete the model forecasts issued from T on, then recompute them. - Tests: judge/screen unit tests; a DB test of refetch, the live hold and the forecast deletion. Checked end to end against RID on a scratch DB. - Docs: DATA_SOURCES (spikes, level-only hours), FLOOD_FORECASTING, rust/README.
This commit is contained in:
1 parent
a610abcdb8
commit
9bf904b23c
8 files changed
+684
-15
No files matched your search
+7
-2
@@ -22,12 +22,17 @@ ping-collector serve [--listen A] [--no-analytics]
|
||||
ping-collector once [--dry-run] # one cycle of every source; --dry-run writes nothing
|
||||
# and compares the RID fetch with the database
|
||||
ping-collector fill-gaps <days|all> # re-fetch RID reports with holes
|
||||
ping-collector refetch FROM [TO] # re-fetch the RID reports of FROM..=TO (YYYY-MM-DD) even
|
||||
# without holes, through the spike screening: corrects
|
||||
# readings RID revised, deletes stored spikes; prints the
|
||||
# catch-up-forecasts command when readings changed
|
||||
ping-collector check-db # connect, create missing tables, row counts
|
||||
ping-collector forecast [--json] [--save] [--now T] [--rain-file F]
|
||||
# forecast from the database and print it
|
||||
ping-collector catch-up-forecasts [--hours N]
|
||||
ping-collector catch-up-forecasts [--hours N] [--redo-from T]
|
||||
# forecast the hours whose readings came late (RID's
|
||||
# overnight batch) as a run then would have; default 336 h
|
||||
# overnight batch) as a run then would have; default 336 h;
|
||||
# --redo-from first deletes the model forecasts from T on
|
||||
ping-collector notify --dry-run # the readings the notifier judges, and what it would send
|
||||
```
|
||||
|
||||
|
||||
Reference in new issue
Block a user