B4L/Northern-Thailand-Ping-River-Monitor
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 15 Wiki Activity

feat: RID large-dam daily collector — Mae Ngat storage/inflow/outflow

Browse Source 
POST app.rid.go.th/reservoir/api/dams (open, archive >=2009) collected
hourly into rid_dams + rid_reservoir_daily; backfill script fetches only
missing days so reruns repair holes and are safe alongside the live
collector. /api/stats counts the new table via an engine fallback that
works when HII collection is disabled. Mae Ngat (DAM_ID 200103) hit 113%
usable capacity with ~19 MCM/day inflow in the Oct 2024 flood — candidate
features for the next retrain.
This commit is contained in:
grabowski
2026-08-13 10:29:34 +07:00
parent 811af1625b
commit 6eafb353b1
7 changed files with 712 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/DATA_SOURCES.md
+3 -3
View File
@@ -183,8 +183,8 @@ Oct 2024 flood). Mae Kuang Udom Thara is the second upstream reservoir.
| Source | What | Access |
|---|---|---|
| `https://lsim.rid.go.th/ForeCast?reservoirid=22` | Mae Ngat daily status/forecast (RID) | Open, scrape |
| `https://app.rid.go.th/reservoir/` | RID reservoir DB, per-reservoir daily detail with date-range URLs | Open, scrape (confirm URL pattern before hard-coding) |
| `https://app.rid.go.th/reservoir/api/dams` | **INGESTED** — daily snapshot of all ~35 large dams (storage/inflow/outflow MCM, % of usable). `POST` with form field `date=YYYY-MM-DD` (empty = today); GET returns 404 "Unknown method." Archive ≥ 2009; `level_msl` (`DMD_Q`) populated in older years only. Mae Ngat = `DAM_ID 200103` — hit 113% usable capacity, ~19 MCM/day inflow, in Oct 2024. Collected daily by `src/rid_reservoir.py` into `rid_dams` + `rid_reservoir_daily`; backfill via `scripts/backfill_rid_reservoir.py` | Open, no auth |
| `https://lsim.rid.go.th/ForeCast?reservoirid=22` | Mae Ngat daily status/forecast (RID) | Open, scrape — timed out from outside RID network when probed 2026-08-13 |
| `https://water.egat.co.th` | EGAT dams (Bhumibol/Sirikit) hourly+daily inflow/outflow/level | Endpoint catalog not public; contact EGAT (0-2436-8186). Only relevant downstream of Bhumibol |
| ThaiWater `/v2/large-dam/*`, `dam_rulecurve/graph` | All large/medium dams incl. hourly | Requires HII API key (§2.2) |
@@ -239,7 +239,7 @@ Oct 2024 flood). Mae Kuang Udom Thara is the second upstream reservoir.
| HII `rain_24h` | ✅ Ingested hourly via `src/hii_collector.py` → `hii_rain_stations` + `hii_rainfall` (Ping-filtered; ~300 stations) | — |
| HII `waterlevel_load` | ✅ Ingested hourly via `src/hii_collector.py` → `hii_wl_stations` + `hii_waterlevel` (125 Ping stations, m MSL; `rid_code` column maps mirrors like `ridhydro_P.1` → `P.1`, `offset_msl` converts MSL → gauge datum) | — |
| HII `waterlevel_graph` | ✅ Backfill via `scripts/backfill_hii_waterlevel.py` → `hii_waterlevel` (hourly MSL + discharge, archive ≥2019; full-year windows per request; upserts never overwrite live-snapshot columns) | Run once on the box: `python scripts/backfill_hii_waterlevel.py` (defaults: 2019-01-01 → today, RID-mirror + key stations; `--stations P.1,P.67`, `--all` for every Ping station) |
| Mae Ngat reservoir (lsim.rid.go.th) | ❌ Not ingested | Add daily storage/release scrape |
| Mae Ngat reservoir (app.rid.go.th) | ✅ Ingested (2026-08-13) | Daily storage/inflow/outflow for all large dams → `rid_reservoir_daily`; candidate model features for next retrain |
| Satellite QPE (GSMaP/IMERG) | ❌ | Backfill training rainfall (GEE) |
| Open-Meteo forecasts | ❌ | Add forecast features (live + 2021 archive for training) |
| HII API key (dams, forecasts) | ❌ | Contact HII for sanctioned access |