master
25
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f4d42c90f4 |
fix: ntfy listens on the Tailscale address; monitor publishes to it directly
Security / Dependency vulnerabilities (push) Successful in 44s
Security / Static analysis (push) Successful in 9s
CI / Format & lint (push) Successful in 10s
CI / Test suite (push) Successful in 26s
Security / License report (push) Successful in 50s
Docs / Validate documentation (push) Successful in 16s
The reverse proxy is a separate VPS on the tailnet, so a loopback-only ntfy was unreachable from it. install_ntfy.sh now binds the host's Tailscale IP (NTFY_LISTEN overrides). New NTFY_PUBLISH_URL: where the monitor POSTs, separate from the public NTFY_SERVER subscribers see, so an alert never waits on DNS or the proxy (first cycle logged 502s from Cloudflare while the domain was not yet proxied). |
||
|
|
777b230baf |
feat: public flood notifications over self-hosted ntfy
CI / Format & lint (push) Successful in 11s
Security / Static analysis (push) Successful in 12s
CI / Test suite (push) Successful in 18s
Docs / Validate documentation (push) Successful in 11s
Security / Dependency vulnerabilities (push) Successful in 1m30s
Security / License report (push) Successful in 47s
Anyone can now get push alerts on their phone without an account: the monitor publishes to an ntfy server (one Go binary, ~30 MB RSS) and subscribers pick topics in the free iOS/Android/web app. Semantics are transitions, never state. One message when a gauge crosses its warning or danger threshold, one all-clear when it drops back (0.10 m hysteresis), nothing while it sits above. A three-day flood is two messages; a quiet season is zero. Topics: ping-warning / ping-danger (basin digest), ping-<station>-warning / -danger, ping-p1-outlook (opt-in: model P(warning within 24 h) at P.1 rises through 50 %, clears below 25 %, message says it is experimental), ping-status (feed stale >= 3 h / recovered). Priority 5 on danger so it rings through Do Not Disturb. src/notify.py runs once per collection cycle in the API process (leader only, after the forecast precompute, same data the dashboard shows). Last-sent state lives in a notification_state table so a restart never re-sends; a failed publish leaves state untouched so the crossing is retried next cycle instead of lost. Off unless NTFY_SERVER is set. Dashboard: a "Get alerts" button (only when configured) opens a panel with the server, per-topic cards, ntfy:// deep links and web links, app store links and a disclaimer. EN + TH. GET /api/notifications feeds it. scripts/install_ntfy.sh: .deb install, server.yml (loopback listen, anonymous read, token-only write scoped to ping-*, 72 h cache, signup/ login/metrics off, tight visitor limits), systemd, user + token, .env. Verified against ntfy 2.28.0: anon publish 403, token publish 200, token on foreign topic 403, anon read 200, and a seeded crossing through the real _notify_transitions path arrived in the topic with priority, tags, click and action button. docs/NOTIFICATIONS.md has the deployment and reverse-proxy notes. Tests: 10 for the state machine (159 total). |
||
|
|
7b31d4d0dd |
feat: "Is the model getting better?" - live verification per model version
CI / Test suite (push) Successful in 22s
CI / Format & lint (push) Successful in 16s
Docs / Validate documentation (push) Successful in 10s
Security / Dependency vulnerabilities (push) Successful in 1m34s
Security / Static analysis (push) Successful in 10s
Security / License report (push) Successful in 13s
src/ml/skill.py joins forecast_history (what each deployed version predicted for the 24 h peak, hourly) to water_measurements (what the river did) and reports per version: verified hours, peak MAE, bias, the persistence baseline (peak = current level), skill = 1 - MAE/persistence, and the same MAE restricted to observed peaks >= 2 m. Only forecasts whose window has elapsed with >= 75 % of hours observed count; a version needs 24 verified hours before it is compared. GET /api/forecast/skill?station_code=P.1&horizon=24 returns it (SWR cached, 15 min). The dashboard's forecast card gains a panel with a one-line verdict (current vs previous version), the per-version table, and a caveat that quiet weeks measure quiet-river accuracy only: the model is judged on flood-onset lead, which the backtests cover. EN + TH. On today's production data: hgb-v3+28b62e5 (369 h, Aug 13 - Sep 1) MAE 15.2 cm, skill -0.05; hgb-v2+f6570ac (224 h, Sep 1 - 11) MAE 12.3 cm, skill 0.36 - the "worse" v2 model scores better on a quieter fortnight, which is exactly why the panel shows the >= 2 m column and the caveat. Tests: 3, sqlite, synthetic. scripts/dev_proxy.py: DEV_PROXY_LOCAL lets a not-yet-deployed endpoint be answered from a local JSON file while everything else goes to prod. |
||
|
|
6f4a86edbb |
fix(dashboard): timestamps are Asia/Bangkok everywhere; stale feed says so
The API emits naive ICT timestamps ("2026-09-12T02:00:00"). The page fed
them to new Date(), which applies the BROWSER's zone: a viewer in Europe
parsed a 02:00 ICT reading as 02:00 CEST, five hours in the future, so
"Last updated" clamped to "0 min ago" forever; a viewer in the Americas
saw thousands of minutes. parseTs() now pins +07:00 on naive strings and
every display formats with timeZone: Asia/Bangkok, so the site shows
river time regardless of where it is opened. Daily/hourly chart buckets
key on the Bangkok calendar day instead of UTC getters.
The tile also gets a real stale state: past 3 h (RID is hourly) it turns
red, reads "Feed stale · last reading <day>, N h ago", and the header
pill switches from LIVE DATA to STALE FEED. Age shows hours past 2 h.
scripts/dev_proxy.py serves the working-copy dashboard with API calls
proxied to water.buildfor.life so browser-side changes can be checked
against live data (and any browser timezone) before deploying.
|
||
|
|
d621aa9ce7 |
eval: quantile heads and fc48 on top of hgb-v3 (rejected/deferred); HII gauge-rain aggregate
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 41s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 17s
Documentation / Validate Documentation (push) Failing after 16s
Documentation / Generate API Documentation (push) Successful in 11s
Documentation / Build Sphinx Documentation (push) Successful in 18s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 3s
Rolling-origin harness gains rise_rain_quantile, rise_rain_quantile_uw, rise_rain_qsigma (L2 point + quantile sigma) and rise_rain_fc48, all opt-in, plus --from-cache for reproducible offline reruns. Results in models/eval_2026-09-12*.json, write-up in docs/FLOOD_FORECASTING.md: - quantile point prediction: better MAE, worse first-alert lead at 5 of 11 events (P.103 2022-08-14 +6h -> +1h) -> rejected - quantile sigma only: Brier within noise (0.0031 -> 0.0029) -> not worth 3x heads - rain_fc48: neutral everywhere except 2024-10-03 P.1 (+21h -> +72h), n=1 -> deferred to after the 2026 season src/ml/hii_rain.py: catchment-mean hourly rain from the ~130 HII gauges in the upper-Ping box and a 24h-sum comparison against Open-Meteo. Not a training feature (table exists only since 2026-08-11, no archive); exposed at GET /api/hii/rainfall/catchment so the two sources' agreement is on record by the time a fold can test it. data._read_cache now skips non-station files in models/cache/ (the shared dir also holds rain_openmeteo / dam_* caches, which crashed the reader). scripts/summarize_eval.py prints per-variant lead/peak-error tables. |
||
|
|
764764e07e |
feat: refuse silent v3->v2 downgrade; monthly retrain timer with staged promote
train_all() now raises RainUnavailableError when use_rain=True and the
Open-Meteo history cannot be loaded, instead of logging a warning and
writing gauge-only (v2) bundles over the deployed v3 set -- which is what
the 2026-09-01 server retrain did unnoticed. --no-rain remains the explicit
way to get v2. CLI exits 2 with a one-line error. Three tests cover the
guard, the opt-out, and the v3 happy path.
scripts/retrain.sh trains into models/.staging, refuses to promote unless
metrics.json shows hgb-v3+ and >=14 trained stations, then renames bundles
into place (previous generation kept in models/.previous). No API restart:
predict.py reloads by mtime on the hourly precompute.
water-monitor-retrain.{service,timer}: 1st of each month 03:30, Persistent,
OMP_NUM_THREADS=4, Nice=15, same sandbox as the API unit. install.sh now
does `uv sync` into .venv (one env rule; removes a stale venv/) and enables
the timer. water-monitor.service in the repo matched neither the deployed
unit nor the uv env; it now does (run.py --web-api, .venv, EnvironmentFile).
|
||
|
|
f6570ac10f |
chore: declutter the repo
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 1m43s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 45s
Documentation / Validate Documentation (push) Failing after 17s
Documentation / Generate API Documentation (push) Successful in 12s
Documentation / Build Sphinx Documentation (push) Successful in 19s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 4s
Removes 26 tracked files that no longer describe or serve the running system, verified one by one against the whole repo (source, tests, docs, README, Makefile, Dockerfile, .gitea workflows, pyproject, packaging spec) plus dynamic-reference paths, before deletion. Root (11): one-off launch/setup write-ups from the project's first weeks that document events which never happened the way they describe — a github.com publication (the remote is self-hosted Gitea) and a 15-minute scheduler (the service runs hourly). Also .gitlab-ci.yml (unused, CI is .gitea/), .env.postgres and setup.py.backup (a placeholder env file and a backup in version control), and the PyInstaller packaging trio build_executable.py / build_simple.py / ping-river-monitor.spec, which bundled docs that no longer exist and is not how this deploys. docs (9): stale guides superseded by DATABASE_DEPLOYMENT_GUIDE, GITEA_WORKFLOWS, FLOOD_FORECASTING and DATA_SOURCES, plus two snapshots (PROJECT_STATUS, PROJECT_STRUCTURE) describing a 4-file src/ that is now 39. scripts (5): one-shot bootstrap tools already run — init_git.sh/.bat, generate_badges.py, migrate_geolocation.py, encode_password.py. Every inbound reference was fixed rather than left dangling: README doc index and migration section, three Makefile targets, the docs.yml summary step, and the GITEA_WORKFLOWS resource list. src/ is deliberately untouched. The audit proposed removing several live modules; verification showed those proposals were mis-scoped and would have broken production. .gitignore now covers the agent tooling dirs, model eval output and editor/shell leftovers — the working tree had collected 56 zero-byte files named after fragments of shell commands. 136 tests pass; production modules import clean. |
||
|
|
382daa7d86 |
perf: backfill one dam per request via the api/dam range endpoint
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 1m6s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
Documentation / Generate API Documentation (push) Successful in 12s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 4s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 34s
Documentation / Validate Documentation (push) Failing after 10s
Documentation / Build Sphinx Documentation (push) Successful in 22s
GET app.rid.go.th/reservoir/api/dam?dam_id&date_start&date_end returns a single dam's whole date range in one response — Mae Ngat's 2009-today archive is ~4 chunked requests instead of the ~2,900 one-day POSTs the all-dams path needs. Field names differ from api/dams and are mapped in parse_dam_range_records, verified equal on spot-checked dates; the range endpoint also carries DMD_ULevel, the reservoir level in m MSL that api/dams stopped publishing after ~2013. scripts/backfill_rid_reservoir.py defaults to the fast per-dam path (--all-dams keeps the full-fleet crawl, --refresh rewrites stored days). Already-stored dates are still skipped, junk values are still bounded, and the consecutive-failure abort still applies. |
||
|
|
28b62e5a36 |
feat: Mae Ngat dam features — built, evaluated, defaulted OFF
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 15s
Documentation / Validate Documentation (push) Failing after 8s
Documentation / Generate API Documentation (push) Successful in 9s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 2s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 27s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
src/ml/dam.py loads rid_reservoir_daily into a leakage-safe hourly frame (daily row visible from 07:00 its own date, ffill capped at 48 h) and is plumbed through features/train/predict/evaluate exactly like rain, gated to the six mainstem stations below the Mae Ngat confluence. The experiment concludes as a documented NEGATIVE result: on the 2024 record-flood backtest every dam-feature subset costs 1-3 h of first-alert lead (13h -> 10-12h) for <=3 cm of peak-error gain, because the daily RID report lags up to 31 h and describes yesterday's benign absorbing reservoir during fast onset. Features therefore default OFF (--dam opt-in on the training and backtest CLIs; rise_rain_dam/rise_dam harness variants, excluded from the default variant set). The ablation also isolated the HII gap-fill as lead-neutral: the acceptance gate holds at 13 h with fill enabled, and docs/img charts are regenerated with the shipping configuration. Full table in docs/FLOOD_FORECASTING.md §5. Review-swarm fixes: evaluate.py skips variants whose feature family is absent instead of crashing the run; --dam forwards --db-url and warns loudly when no dam history loads; an empty DB result can no longer wipe a good dam cache; run-level metrics version claims v4 only when a dam station is actually in the set. |
||
|
|
6eafb353b1 |
feat: RID large-dam daily collector — Mae Ngat storage/inflow/outflow
POST app.rid.go.th/reservoir/api/dams (open, archive >=2009) collected hourly into rid_dams + rid_reservoir_daily; backfill script fetches only missing days so reruns repair holes and are safe alongside the live collector. /api/stats counts the new table via an engine fallback that works when HII collection is disabled. Mae Ngat (DAM_ID 200103) hit 113% usable capacity with ~19 MCM/day inflow in the Oct 2024 flood — candidate features for the next retrain. |
||
|
|
160617e87b |
feat: openmeteo_rain 2021+ backfill entry point
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 26s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
Documentation / Validate Documentation (push) Failing after 7s
Documentation / Generate API Documentation (push) Successful in 9s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 12s
Documentation / Build Sphinx Documentation (push) Successful in 18s
Documentation / Documentation Summary (push) Successful in 2s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 0s
rain.backfill_db pushes the full cached Open-Meteo archive into the openmeteo_rain table in 5k-row idempotent upsert chunks; scripts/backfill_rain_db.py is the thin CLI (DB from Config/.env or --db-url). Safe to re-run and safe alongside the hourly live writer. |
||
|
|
df0ae8cda3 |
feat: hgb-v3 — Open-Meteo rain features clear the 12h warning gate
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 13s
Documentation / Validate Documentation (push) Failing after 7s
Documentation / Generate API Documentation (push) Successful in 8s
Documentation / Build Sphinx Documentation (push) Successful in 17s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 0s
Documentation / Documentation Summary (push) Successful in 2s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 24s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
The rolling-origin harness (models/eval_rain.json) showed catchment rain halving flood-year Brier scores, cutting flood-regime MAE 20-40%, and extending the hard 2024 leads (+6h -> +11h at P.1, +10h -> +19h at P.103). Ported: train_all loads the catchment-mean series (use_rain / --no-rain to opt out; without it bundles train as v2), predict fetches live rain hourly and passes an empty series on failure so rain-trained bundles serve with NaN features instead of tripping the feature guard, and the leader worker persists hourly per-point + catchment-mean rows to a new openmeteo_rain table. Regenerated backtest: the 2024 record flood now gets a 13-HOUR WARNING (alert 04:00 vs 17:00 crossing, river at 2.9m at alert time) — the >=12h acceptance gate PASSES for the first time. Journey on that crossing: v1 -18h, v2 +6h, v3 +13h. The marginal 2025 double-crest trades its artifact +46h latch for a calibrated +2h with zero false alarms. P.1 MAE 4.9/7.2/8.7 cm at 6/12/24h. Docs updated throughout. |
||
|
|
21e9d2e114 |
feat: hgb-v2 — regression heads predict rise, recovering flood warning lead
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 15s
Documentation / Validate Documentation (push) Failing after 9s
Documentation / Generate API Documentation (push) Successful in 8s
Documentation / Build Sphinx Documentation (push) Successful in 17s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 28s
Documentation / Documentation Summary (push) Successful in 2s
Rolling-origin evaluation (5 monsoon folds x 4 variants, P.1 + P.103; results in models/eval_variants.json) showed the absolute-level target alerting AT the crossing on essentially every event, while the rise target (future max - current level, level added back at serving) gives +6h on the hard 2024 crossings, +45h in 2025, fewer false alarms than weighted/quantile variants, and ~11% better MAE. Weighted and quantile variants rejected: more false alarms, no Brier-score calibration gain. Ported to production: train.py fits rise in both eval and refit passes (sigma/metrics computed in absolute space), bundles stamped hgb-v2 with regression_target='rise', predict.py adds the level back for v2 and stays compatible with v1 bundles, backtest_render.py mirrors the same math. Regenerated backtest charts: 2024 first alert 11:00 24 Sep (6h BEFORE the 17:00 crossing, was 18h after), 2025 alert 45h ahead, and the record-peak underprediction is gone (rise models can exceed the training max). The >=12h acceptance gate still fails honestly at +6h — closing that needs rainfall inputs. New P.1 MAE 5.0/7.2/9.4 cm at 6/12/24h; docs updated throughout. |
||
|
|
a0086086a2 |
feat: rolling-origin event-aware evaluation harness for model variants
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 24s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 13s
Documentation / Validate Documentation (push) Failing after 8s
Documentation / Generate API Documentation (push) Successful in 8s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 2s
One fold per monsoon season (train <= 30 Apr, test Jun-Nov, 2021-2025) replaces the single fixed holdout that contained only ~4 warning events. Metrics are what matters operationally: sustained first-alert lead vs each observed 3.70m crossing (two consecutive alerting samples required; lookback floored at the previous event's end so multi-peak floods can't launder lead credit), peak error from the prediction actually issued 24h before the peak (3h match tolerance, null on outages), false-alarm episodes (12h gap tolerance), MAE / flood-regime MAE, and a Brier score on warning exceedance — included because sigma cancels algebraically in any p>=0.5 alert metric, so lead times compare predictors while Brier compares uncertainty models. Variants: baseline_abs (current), rise (target = future max - current level), rise_weighted (flood-regime sample weights 1x->5x), and rise_quantile (q50/q90 heads, spread-implied sigma). Harness verified by a 3-agent adversarial review (features bit-identical across fold cutoffs; three metric flaws found and fixed before first use). Also: features.build_labels/build_matrix gain stats_end so the rescue quantile is computed from pre-cutoff data only, closing the label- construction leak flagged in the earlier ML review. |
||
|
|
1ec5cfb4df |
perf: gzip responses; multi-worker serving with single collection leader
Documentation / Validate Documentation (push) Failing after 8s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 23s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 12s
Documentation / Generate API Documentation (push) Successful in 9s
Documentation / Documentation Summary (push) Successful in 2s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 0s
GZipMiddleware (min 500 bytes) compresses the dashboard HTML ~4x and station JSON up to ~100x, end-to-end through the Caddy TLS terminator — production load testing showed the deployment is bandwidth-bound once the response caches hit, so compression is the capacity lever. WEB_WORKERS (default 2) runs uvicorn multi-process via the app import string. Every worker executes the lifespan, so a localhost lock port (COLLECTION_LEADER_PORT, default 8901) elects exactly one background-collection leader per machine — RID/HII polling stays once-per-cycle instead of once-per-worker; the lock releases with the process. Locust clients now send Accept-Encoding so future runs measure compressed transfer, as browsers do. |
||
|
|
0005f7dce1 |
feat: codified backtests, honest docs, belt-and-braces serving, perf fixes
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 13s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 23s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
Documentation / Validate Documentation (push) Failing after 7s
Documentation / Generate API Documentation (push) Successful in 8s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 3s
Retrained on the gap-filled DB (592k -> 976k rows) and re-examined the flood backtests, now reproducible via scripts/backtest_render.py (renders the three docs/img charts and gates on a >=12h 2024 first-alert lead — currently failing by design and documented as such). Findings, all documented in FLOOD_FORECASTING.md: the true 2024 crossing was 24 Sep 17:00 (8h earlier than recorded; confirmed against the independent HII sensor), the historical 24h-warning claim was partly a missing-data artifact, and retrained warn classifiers collapse on the filled grid (P.1 24h PR-AUC 0.900 -> 0.288) while regression MAE improves (11.3 -> 10.5 cm). Serving therefore becomes max(classifier, sigmoid(regression)) so alerting is never worse than the regression path; metrics table, head-gating tiers, honest-limits and runbook expectations all updated to the current model (hgb-v1+d2d0e65). Perf, from Locust load testing (scripts/locustfile.py + load_test.py): single-flight lock around /forecast inference (concurrent cache misses previously each ran ~18s inference and starved the shared thread pool; 200-user run after: 105 rps, 0.01% errors), and /measurements/latest + /health moved off the event loop (synchronous DB/network calls in async handlers were stalling every request under load). |
||
|
|
d72496f404 |
feat: backfill hii_waterlevel from the HII waterlevel_graph archive
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 31s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 0s
Documentation / Documentation Summary (push) Successful in 2s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 13s
Documentation / Validate Documentation (push) Failing after 9s
Documentation / Generate API Documentation (push) Successful in 9s
Documentation / Build Sphinx Documentation (push) Successful in 15s
scripts/backfill_hii_waterlevel.py walks the api-v3 waterlevel_graph endpoint (hourly wl_msl + discharge, archive back to ~2019) in full-year windows per station and upserts into hii_waterlevel. Defaults to the RID-mirror and key stations; --stations/--all/--start/--end/--chunk-days override. History upserts touch only wl_msl and discharge so colliding live-snapshot rows keep storage_percent/situation_level. Idempotent and safe to re-run. |
||
|
|
4358d52d55 |
feat: ML flood-event forecasting from 8 years of gauge history
Security & Dependency Updates / Dependency Security Scan (push) Successful in 1m8s
Security & Dependency Updates / License Compliance (push) Successful in 25s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 20s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 17s
Security & Dependency Updates / Security Summary (push) Successful in 9s
Add src/ml/ package predicting, per station and per 6/12/24 h horizon, the probability of exceeding warning (3.0 m) and danger (4.5 m) levels plus expected peak level, trained on the 592k-row PostgreSQL history: - features.py: hourly grid with coverage gating and no future leakage; upstream stations enter at empirically measured travel-time lags (P.20 +17h ... P.103 +1h vs P.1); hour-of-day deliberately excluded (it encodes the scrape schedule, not hydrology) - train.py: HistGradientBoosting regression + warn/danger classifier heads per station x horizon, >=30-positives gate with calibrated sigmoid-on-regression fallback, strict temporal splits, per-event lead-time evaluation; guards against sklearn 1.9.0 crash on degenerate feature columns - predict.py: bundle loading with feature-name checks, heuristic fallback tier, get_latest_forecasts() for the API; raises when no models are trained so the endpoint 503s instead of serving persistence output as forecasts - data.py: Postgres-first loader (FLOOD_ML_DB_URL override), HTTP API fallback (flagged: that path backfills synthetic discharge), csv.gz cache - /forecast endpoint (15-min TTL cache) + dashboard flood-risk panel (hidden until models exist) - docs/FLOOD_FORECASTING.md: full system doc with measured deployment numbers (~335 MB RSS, CPU negligible, ~6 min full retrain) and retraining policy Validation: out-of-sample backtest of the record 2024 flood season (train <= Aug 2024) alerted 24-48 h ahead of the Oct 5 peak; 2025-26 test split: P.1 6h PR-AUC 0.974, recall 98.3% at 1% false-alarm rate. Also: fix P.81 station coordinates (was Ban Pong/Ratchaburi, 493 km out of basin; now 18.6936 N 99.0819 E per RID station page), pin scikit-learn==1.9.0 and numpy<2, gitignore model artifacts (~100 MB, train on the server via scripts/train_flood_model.py). |
||
|
|
ce31a5254e |
Harden install.sh per security review
- .env now chmod 0600 and APP_DIR chmod 0750 after chown, so the Matrix token and DB credentials are not world-readable. - uv auto-install (curl | sh as root) is now opt-in via AUTO_INSTALL_UV=1 and pins a specific uv version; otherwise the script requires uv to be pre-installed and fails with instructions, avoiding unattended remote code execution as root. |
||
|
|
ab8a10dd75 |
Add install.sh and fix service unit placeholder
- scripts/install.sh: one-command hardened deploy (creates the water-monitor system user, deploys to /opt, builds a uv-managed venv, installs and enables the systemd unit). Idempotent; excludes .env/*.db/stations.json from sync so runtime state is preserved. - Fix placeholder Documentation= URL in water-monitor.service. - README: document the script as the primary systemd install path, with manual steps kept as a fallback. |
||
|
|
6c7c128b4d |
Major refactor: Migrate to uv, add PostgreSQL support, and comprehensive tooling
- **Migration to uv package manager**: Replace pip/requirements with modern pyproject.toml - Add pyproject.toml with complete dependency management - Update all scripts and Makefile to use uv commands - Maintain backward compatibility with existing workflows - **PostgreSQL integration and migration tools**: - Enhanced config.py with automatic password URL encoding - Complete PostgreSQL setup scripts and documentation - High-performance SQLite to PostgreSQL migration tool (91x speed improvement) - Support for both connection strings and individual components - **Executable distribution system**: - PyInstaller integration for standalone .exe creation - Automated build scripts with batch file generation - Complete packaging system for end-user distribution - **Enhanced data management**: - Fix --fill-gaps command with proper method implementation - Add gap detection and historical data backfill capabilities - Implement data update functionality for existing records - Add comprehensive database adapter methods - **Developer experience improvements**: - Password encoding tools for special characters - Interactive setup wizards for PostgreSQL configuration - Comprehensive documentation and migration guides - Automated testing and validation tools 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> |
||
|
|
17a716fcd0 |
Version bump: 3.1.2 3.1.3 (Force new build)
Release - Northern Thailand Ping River Monitor / Create Release (push) Successful in 7s
Security & Dependency Updates / Dependency Security Scan (push) Successful in 35s
Security & Dependency Updates / Check for Dependency Updates (push) Has been cancelled
Security & Dependency Updates / Code Quality Metrics (push) Has been cancelled
Security & Dependency Updates / Security Summary (push) Has been cancelled
Security & Dependency Updates / License Compliance (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.11) (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.12) (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.9) (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Build Release Images (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Security Scan (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Deploy Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Validate Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Notify Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.10) (push) Has been cancelled
Version Updates: - Core application: src/__init__.py, src/main.py, src/web_api.py - Package configuration: setup.py - Documentation: README.md, docs/GITEA_WORKFLOWS.md - Workflows: .gitea/workflows/docs.yml, .gitea/workflows/release.yml - Scripts: generate_badges.py, init_git scripts - Tests: test_integration.py - Deployment docs: GITEA_SETUP_SUMMARY.md, DEPLOYMENT_CHECKLIST.md Purpose: - Force new build process after workflow fixes - Test updated security.yml without YAML errors - Verify setup.py robustness improvements - Trigger clean CI/CD pipeline execution All version references synchronized at v3.1.3 Ready for new build and deployment testing |
||
|
|
40aef686af |
Fix: Replace GitHub checkout with Gitea checkout + Version bump
Release - Northern Thailand Ping River Monitor / Create Release (push) Failing after 1s
Release - Northern Thailand Ping River Monitor / Build Release Images (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Test Release Build (3.10) (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Test Release Build (3.11) (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Test Release Build (3.12) (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Test Release Build (3.9) (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Security Scan (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Deploy Release (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Validate Release (push) Has been skipped
Security & Dependency Updates / Dependency Security Scan (push) Failing after 1s
Security & Dependency Updates / Docker Security Scan (push) Failing after 1s
Security & Dependency Updates / License Compliance (push) Failing after 1s
Security & Dependency Updates / Check for Dependency Updates (push) Failing after 1s
Security & Dependency Updates / Code Quality Metrics (push) Failing after 1s
Release - Northern Thailand Ping River Monitor / Notify Release (push) Successful in 1s
Security & Dependency Updates / Security Summary (push) Failing after 3s
Checkout Action Migration: - Replace all 'actions/checkout@v4' with 'https://gitea.com/actions/checkout' - Fixes 'Bad credentials' errors when workflows try to access GitHub API - Native Gitea checkout action eliminates authentication issues - Applied across all 4 workflow files (CI, Security, Release, Docs) Version Increment: 3.1.1 3.1.2 - Core application version updates - Web API version synchronization - Documentation version alignment - Badge and release example updates Problem Solved: - Workflows no longer attempt GitHub API calls - Gitea-native checkout action handles repository access properly - Eliminates 'Retrieving the default branch name' failures - Cleaner workflow execution without authentication errors Files Updated: - 4 workflow files: checkout action replacement - 13 files: version number updates - Consistent v3.1.2 across all components Benefits: - Workflows will now run successfully in Gitea - No more GitHub API authentication failures - Native Gitea action compatibility - Ready for successful CI/CD pipeline execution |
||
|
|
19e182c53b |
Version bump: 3.1.0 3.1.1
Release - Northern Thailand Ping River Monitor / Create Release (push) Successful in 6s
Security & Dependency Updates / Dependency Security Scan (push) Successful in 19s
Security & Dependency Updates / Docker Security Scan (push) Successful in 1m12s
Security & Dependency Updates / License Compliance (push) Successful in 11s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 14s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 9s
Release - Northern Thailand Ping River Monitor / Test Release Build (3.10) (push) Failing after 1m17s
Release - Northern Thailand Ping River Monitor / Test Release Build (3.11) (push) Failing after 23s
Release - Northern Thailand Ping River Monitor / Test Release Build (3.9) (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Build Release Images (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Security Scan (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Deploy Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Validate Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Notify Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.12) (push) Has been cancelled
Security & Dependency Updates / Security Summary (push) Has been cancelled
Version Updates: - Core application version (src/__init__.py) - Web API version (src/web_api.py) - Main application logging (src/main.py) - Package setup version (setup.py) - Documentation generation (docs workflow) - Release workflow example version - Badge generation script - Integration test version display - README.md badge version - Setup and deployment documentation - Git initialization scripts Patch Release (3.1.1): - Workflow token migration fixes (GITHUB_TOKEN GH_TOKEN) - Pip installation warning elimination - Improved workflow reliability and logging - Better Gitea compatibility - Enhanced error handling and validation Files Updated: - 13 files with version references updated - Consistent versioning across all components - Ready for release tagging and deployment |
||
|
|
af62cfef0b |
Initial commit: Northern Thailand Ping River Monitor v3.1.0
Security & Dependency Updates / Dependency Security Scan (push) Successful in 29s
Security & Dependency Updates / Docker Security Scan (push) Failing after 53s
Security & Dependency Updates / License Compliance (push) Successful in 13s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 19s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 11s
Security & Dependency Updates / Security Summary (push) Successful in 7s
Features: - Real-time water level monitoring for Ping River Basin (16 stations) - Coverage from Chiang Dao to Nakhon Sawan in Northern Thailand - FastAPI web interface with interactive dashboard and station management - Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics) - Comprehensive monitoring with health checks and metrics collection - Docker deployment with Grafana integration - Production-ready architecture with enterprise-grade observability CI/CD & Automation: - Complete Gitea Actions workflows for CI/CD, security, and releases - Multi-Python version testing (3.9-3.12) - Multi-architecture Docker builds (amd64, arm64) - Daily security scanning and dependency monitoring - Automated documentation generation - Performance testing and validation Production Ready: - Type safety with Pydantic models and comprehensive type hints - Data validation layer with range checking and error handling - Rate limiting and request tracking for API protection - Enhanced logging with rotation, colors, and performance metrics - Station management API for dynamic CRUD operations - Comprehensive documentation and deployment guides Technical Stack: - Python 3.9+ with FastAPI and Pydantic - Multi-database architecture with adapter pattern - Docker containerization with multi-stage builds - Grafana dashboards for visualization - Gitea Actions for CI/CD automation - Enterprise monitoring and alerting Ready for deployment to B4L infrastructure! |