feat(rid): screen one-hour spikes; refetch and redo-from to repair the store
RID's report sometimes carries a wrong value for one hour and corrects it later. P.1 read 10.6 m at 2026-10-25 00:00 between 1.04 and 1.07 (RID now says 1.06); the store kept it, because a report is re-fetched only while it has holes, and the model issued a 10.8 m forecast for that hour. - rid::judge / rid::screen: a reading that leaves the previous one (within 3 h) by more than 1 m (2 m on the flashy tributaries P.76, P.77, P.81, P.82, P.84, P.85, P.87) while the next reading comes back near the previous level is a spike: not stored, a stored copy is deleted. Limbs and steps the next reading confirms are kept; the newest reading waits for the next one when it jumps more than twice the limit. On 2018-2026 history: 16 spikes found (slipped decimals, sign flips, -5/-6/-9 m placeholders), no genuine change rejected. - Every store goes through it (rid::store: run_cycle, fill-gaps, refetch). - `ping-collector refetch FROM [TO]`: re-fetch reports regardless of holes (corrects revised readings, fills the discharge of level-only evening hours, removes stored spikes) and print the catch-up command when readings changed. - `catch-up-forecasts --redo-from T`: delete the model forecasts issued from T on, then recompute them. - Tests: judge/screen unit tests; a DB test of refetch, the live hold and the forecast deletion. Checked end to end against RID on a scratch DB. - Docs: DATA_SOURCES (spikes, level-only hours), FLOOD_FORECASTING, rust/README.
This commit is contained in:
1 parent
a610abcdb8
commit
9bf904b23c
8 files changed
+684
-15
No files matched your search
@@ -643,7 +643,10 @@ Open-Meteo's estimate of rain that has since fallen, as in training, where
|
||||
the forward rain is observed rain. Such rows are written only where no model
|
||||
row exists; `created_at` tells them apart (more than an hour after `as_of`).
|
||||
`ping-collector catch-up-forecasts --hours N` does the same further back (up
|
||||
to 60 days).
|
||||
to 60 days); with `--redo-from T` it first deletes the model forecasts issued
|
||||
from T on, to recompute them after `ping-collector refetch` corrected the
|
||||
readings they came from (a bad reading reaches the features for 72 h through
|
||||
`roll_max_72`).
|
||||
|
||||
A model row in `forecast_history` is never replaced by a heuristic one. While
|
||||
RID is silent overnight each hourly run re-issues the 21:00 reading, and once
|
||||
|
||||
Reference in new issue
Block a user