9bf904b23c4db8ca3405fef5f6c521288b2d98cb
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 grabowski
|
cf10f2e4c8 |
feat(dashboard): console phase 4 - 2024 replay, simulation, replaces the old page
The console is now the dashboard at / (src/static/dashboard.html); the old page and console.html are gone. - Replay of the October 2024 flood (flood-2024-all.json): the page's clock, verdict, list, markers, flood zones and panel follow the replay; the panel charts the season on fixed axes with the 2024 model's 24 h peak. Hourly while the model alerts or P.1 is above 3.50 m, faster through quiet days; pause and drag. Rain, basin stations and the 5-minute refresh are off during the replay (today's data). - ?demo_level= and ?demo_rise= simulate P.1. A violet REPLAY / amber SIMULATION bar, the top bar's KPI and the P.1 panel say so; the mode is drawn from state.mode alone, so a refresh or language switch cannot relabel either as live. - History: rows without discharge_percent are ThaiWater's copy of the gauge (night hours RID publishes late), with a different rating (P.1: 26 against RID's 54 m3/s); their discharge is not drawn for gauges with RID discharge. The discharge strip has headroom so its label stays clear of the line. - Swap: the old page's title, description, Open Graph and JSON-LD tags; no noindex; tests/test_dashboard.py = console tests + the old checks that still apply (one P.1 level, SEO files) + a replay/simulation labelling test. - Docs: README screenshot (docs/img/dashboard.jpg, replacing two unused images), CLAUDE.md, CONTRIBUTING.md, FLOOD_FORECASTING.md (where the model shows), DASHBOARD_REDESIGN.md (phase 4 done, mode rules). Deploy needs a restart: the server reads dashboard.html at start. |
||
|
grabowski
|
71cc1b0ca7 |
feat(dashboard): console phase 2 - self-hosted basemap, basin stations, markers
- Basemap: src/static/basemap/ping-basin.pmtiles, a Protomaps extract of the Ping basin (zoom 0-12, 15 MB, OpenStreetMap data), drawn by protomaps-leaflet 5.1.0 (unpkg, SRI) in its light/dark style with labels in the page language. It replaces the dimmed OSM raster tiles, whose baked-in labels were the noisiest part of the screen; OSM stays as the fallback. The browser reads the file with range requests (ServeDir answers 206; dev_proxy.py now does too). The map stays within the basin. - Basin stations: a second list next to the RID gauges and a map layer of diamonds, from /api/hii/waterlevel/latest (falling back to /sensors/thaiwater), coloured by ThaiWater's bank-height classes (over the bank = danger, 70-100 % = watch). Gauges RID already lists appear once. The panel shows level (m MSL), % of bank height, distance to and height of the bank top, gauge reading and agency; selection in the URL (#hii-<id>). - custom-markers.json as pins from zoom 12, with the flood zone they are in. Refresh instructions for the basemap in src/static/basemap/README.md. |
||
|
grabowski
|
301a629b42 |
feat(dashboard): the console, phase 1 of the redesign (preview at /static/console.html)
The user chose direction B of the redesign: a full-screen operations console.
It is built next to the current dashboard, served by ServeDir at
/static/console.html (noindex, links back to /), and replaces dashboard.html
once it covers every feature (docs/DASHBOARD_REDESIGN.md, phase 4).
Phase 1:
- top bar: the Chiang Mai verdict (P.1 level + 24 h forecast, the current
dashboard's rule), gauges reporting, highest 24 h flood chance, data age
("overnight pause" instead of "stale" during RID's nightly gap, as in the
notifier), EN/TH, light/dark (dark default), Get alerts
- gauge list: one status rule for list, map and panel (normal, watch within
0.5 m or >= 25 % chance, warning, danger, no data), margin to the warning
level, 72 h sparkline, 6 h change; filter; sort by river order, closest to
the warning level, rising; arrow keys
- map: Ping mainstem and tributaries, status markers with code + level,
layers Rain 24 h (on by default only with heavy rain) and Flood zones
(auto-shown while P.1 is at a stage), one legend
- station panel: level, change, discharge (last RID value when the newest
hour is level-only), warning/danger levels with margins, capacity, level
scale with now and 24 h peak, 72 h chart with the forecast peaks, forecast
table with the model and issue time, the P.1 inundation stages, data links
- alerts drawer with the ntfy topics; selection in the URL (#P.84); refresh
every 5 minutes
- Inter and Noto Sans Thai self-hosted (src/static/fonts, 75 KB, OFL), no
emoji, Thai first, Bangkok time
tests/test_console.py: routed fetch paths, EN/TH parity, translated aria
labels, no emoji, fonts and licences, noindex. dev_proxy.py serves
src/static with real content types (it sent HTML as octet-stream).
|
||
|
grabowski
|
7e642555b4 |
chore: remove the Python service; Python is the training stack only
Phase 4 of docs/RUST_REWRITE.md. Since 2026-10-18 ping-collector (rust/) collects, forecasts, notifies (ntfy + Matrix) and serves the website/API; nothing in production ran the Python service code any more. Removed (~17k lines): the FastAPI app (web_api, schemas, health_check, metrics, postgres_history, forecast_history, thaiwater), the Python RID collector and CLI (water_scraper_v3, main, run.py), the notifier and Matrix alerting (notify, alerting), the multi-backend adapters (SQLite, MySQL, InfluxDB, VictoriaMetrics) with demo databases, models/validators/exceptions/ logging/rate limiter, ml/skill.py (served by Rust now), the migration tooling (api_parity, collector_parity, e2e_notify, install_collector.sh, gen_notify_cases.py, spikes/rust-forecast), Docker/compose/Grafana files, sql/, the release workflow and the docs for all of that. Their tests went with them. Kept in Python: src/ml (training, evaluation harness, export; predict.py as the reference the Rust forecaster is tested against), the HII and RID-dam clients/stores for the archive backfills (their live collectors removed), the backfill scripts, retrain.sh + timer, forecast_parity.py. - PostgreSQL only: src/config.py is database_url() from POSTGRES_* / .env; the stores lost their sqlite/mysql branches (and the 2026-08 column migration). The database tests run against a throwaway schema from TEST_DATABASE_URL (tests/conftest.py pg_url, skipped without it), as the Rust tests do; CI gets a PostgreSQL service. 100 tests (66 without a DB). - The notifier settings moved from src/notify.py into src/ml/export.py; flood_config.json regenerates byte-identical. - tests/test_dashboard.py checks every path the dashboard fetches against ping-collector's router. - Dependencies: fastapi, uvicorn, pydantic, influxdb, pymysql, psutil, schedule, sphinx, pyinstaller, mypy, pytest-asyncio dropped; python-dotenv 1.1.1 explicit (it came in through uvicorn[standard]). - Formatting contract extended to scripts/ and tests/ (reformatted once). - scripts/install.sh is the one installer (fresh host or update, in place in a checkout without touching its ownership): binary, units, uv sync, .env defaults, removes the retired water-monitor.service and any old --alert-check cron job, dry run + forecast check, restart, readiness wait. - Docs, CLAUDE.md, CONTRIBUTING.md, .env.example, Makefile, unit comments and Rust doc comments updated to the Rust service. |
||
|
grabowski
|
7b31d4d0dd |
feat: "Is the model getting better?" - live verification per model version
CI / Test suite (push) Successful in 22s
CI / Format & lint (push) Successful in 16s
Docs / Validate documentation (push) Successful in 10s
Security / Dependency vulnerabilities (push) Successful in 1m34s
Security / Static analysis (push) Successful in 10s
Security / License report (push) Successful in 13s
src/ml/skill.py joins forecast_history (what each deployed version predicted for the 24 h peak, hourly) to water_measurements (what the river did) and reports per version: verified hours, peak MAE, bias, the persistence baseline (peak = current level), skill = 1 - MAE/persistence, and the same MAE restricted to observed peaks >= 2 m. Only forecasts whose window has elapsed with >= 75 % of hours observed count; a version needs 24 verified hours before it is compared. GET /api/forecast/skill?station_code=P.1&horizon=24 returns it (SWR cached, 15 min). The dashboard's forecast card gains a panel with a one-line verdict (current vs previous version), the per-version table, and a caveat that quiet weeks measure quiet-river accuracy only: the model is judged on flood-onset lead, which the backtests cover. EN + TH. On today's production data: hgb-v3+28b62e5 (369 h, Aug 13 - Sep 1) MAE 15.2 cm, skill -0.05; hgb-v2+f6570ac (224 h, Sep 1 - 11) MAE 12.3 cm, skill 0.36 - the "worse" v2 model scores better on a quieter fortnight, which is exactly why the panel shows the >= 2 m column and the caveat. Tests: 3, sqlite, synthetic. scripts/dev_proxy.py: DEV_PROXY_LOCAL lets a not-yet-deployed endpoint be answered from a local JSON file while everything else goes to prod. |
||
|
grabowski
|
6f4a86edbb |
fix(dashboard): timestamps are Asia/Bangkok everywhere; stale feed says so
The API emits naive ICT timestamps ("2026-09-12T02:00:00"). The page fed
them to new Date(), which applies the BROWSER's zone: a viewer in Europe
parsed a 02:00 ICT reading as 02:00 CEST, five hours in the future, so
"Last updated" clamped to "0 min ago" forever; a viewer in the Americas
saw thousands of minutes. parseTs() now pins +07:00 on naive strings and
every display formats with timeZone: Asia/Bangkok, so the site shows
river time regardless of where it is opened. Daily/hourly chart buckets
key on the Bangkok calendar day instead of UTC getters.
The tile also gets a real stale state: past 3 h (RID is hourly) it turns
red, reads "Feed stale · last reading <day>, N h ago", and the header
pill switches from LIVE DATA to STALE FEED. Age shows hours past 2 h.
scripts/dev_proxy.py serves the working-copy dashboard with API calls
proxied to water.buildfor.life so browser-side changes can be checked
against live data (and any browser timezone) before deploying.
|