Commit Graph
10 Commits
Author SHA1 Message Date
grabowski 5ad8e4eac3 ci: green pipelines that check what exists; one formatting contract
The Test Suite job failed on every push since the black check was added
because the tree had never been formatted, and pre-commit said 120
columns while CI ran black's default 88. pyproject.toml now carries
[tool.black] / [tool.isort] (88, black profile) as the single source;
pre-commit reads it; `make format` applied it (13 files, whitespace only,
146 insertions / 128 deletions, tests unchanged at 146 passed).

ci.yml: lint (black, isort, flake8 hard errors) + pytest. The Docker
registry push, VictoriaMetrics integration test, staging/production
deploy and Apache-Bench jobs were template scaffolding for hosts and
registries that do not exist; production is a systemd unit updated by
git pull. Removed rather than left permanently skipped.

docs.yml: the "Check markdown links" step curl'd every URL in every .md
and failed on localhost examples and the Tailscale IP, and the Sphinx
jobs built artifacts nobody read. Replaced by two checks that mean
something: relative links/images in README, CONTRIBUTING and docs/
resolve inside the repo, and the FastAPI OpenAPI schema exports with
the documented endpoints present (uploaded as an artifact).
2026-09-11 23:05:37 +02:00
grabowski ce08312c0f docs: public dashboard URL (water.buildfor.life) replaces the Tailscale IP everywhere
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 32s
Documentation / Documentation Summary (push) Successful in 3s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 15s
Documentation / Validate Documentation (push) Failing after 16s
Documentation / Generate API Documentation (push) Successful in 11s
Documentation / Build Sphinx Documentation (push) Successful in 18s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
2026-09-11 22:38:13 +02:00
grabowski 764764e07e feat: refuse silent v3->v2 downgrade; monthly retrain timer with staged promote
train_all() now raises RainUnavailableError when use_rain=True and the
Open-Meteo history cannot be loaded, instead of logging a warning and
writing gauge-only (v2) bundles over the deployed v3 set -- which is what
the 2026-09-01 server retrain did unnoticed. --no-rain remains the explicit
way to get v2. CLI exits 2 with a one-line error. Three tests cover the
guard, the opt-out, and the v3 happy path.

scripts/retrain.sh trains into models/.staging, refuses to promote unless
metrics.json shows hgb-v3+ and >=14 trained stations, then renames bundles
into place (previous generation kept in models/.previous). No API restart:
predict.py reloads by mtime on the hourly precompute.

water-monitor-retrain.{service,timer}: 1st of each month 03:30, Persistent,
OMP_NUM_THREADS=4, Nice=15, same sandbox as the API unit. install.sh now
does `uv sync` into .venv (one env rule; removes a stale venv/) and enables
the timer. water-monitor.service in the repo matched neither the deployed
unit nor the uv env; it now does (run.py --web-api, .venv, EnvironmentFile).
2026-09-11 21:37:11 +02:00
grabowski f6570ac10f chore: declutter the repo
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 1m43s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 45s
Documentation / Validate Documentation (push) Failing after 17s
Documentation / Generate API Documentation (push) Successful in 12s
Documentation / Build Sphinx Documentation (push) Successful in 19s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 4s
Removes 26 tracked files that no longer describe or serve the running
system, verified one by one against the whole repo (source, tests, docs,
README, Makefile, Dockerfile, .gitea workflows, pyproject, packaging spec)
plus dynamic-reference paths, before deletion.

Root (11): one-off launch/setup write-ups from the project's first weeks
that document events which never happened the way they describe — a
github.com publication (the remote is self-hosted Gitea) and a 15-minute
scheduler (the service runs hourly). Also .gitlab-ci.yml (unused, CI is
.gitea/), .env.postgres and setup.py.backup (a placeholder env file and a
backup in version control), and the PyInstaller packaging trio
build_executable.py / build_simple.py / ping-river-monitor.spec, which
bundled docs that no longer exist and is not how this deploys.

docs (9): stale guides superseded by DATABASE_DEPLOYMENT_GUIDE,
GITEA_WORKFLOWS, FLOOD_FORECASTING and DATA_SOURCES, plus two snapshots
(PROJECT_STATUS, PROJECT_STRUCTURE) describing a 4-file src/ that is now 39.

scripts (5): one-shot bootstrap tools already run — init_git.sh/.bat,
generate_badges.py, migrate_geolocation.py, encode_password.py.

Every inbound reference was fixed rather than left dangling: README doc
index and migration section, three Makefile targets, the docs.yml summary
step, and the GITEA_WORKFLOWS resource list.

src/ is deliberately untouched. The audit proposed removing several live
modules; verification showed those proposals were mis-scoped and would
have broken production.

.gitignore now covers the agent tooling dirs, model eval output and
editor/shell leftovers — the working tree had collected 56 zero-byte
files named after fragments of shell commands.

136 tests pass; production modules import clean.
2026-08-14 13:45:03 +07:00
grabowski 1845ef7203 feat: hourly HII/ThaiWater rainfall + backup water-level collection
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 37s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 16s
Documentation / Validate Documentation (push) Failing after 9s
Documentation / Generate API Documentation (push) Successful in 10s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 0s
Documentation / Documentation Summary (push) Successful in 2s
Poll the open api-v3.thaiwater.net public endpoints (rain_24h,
waterlevel_load), filter to the Ping basin, and persist to new
hii_rain_stations/hii_rainfall and hii_wl_stations/hii_waterlevel tables
(auto-created; sqlite/postgresql/mysql). Water levels stay in their own
tables since HII reports m MSL from a different station set; rid_code
maps mirrors like ridhydro_P.1 to P.1 and offset_msl converts MSL to
gauge datum. Runs every scraping cycle in web-api and continuous modes
(hourly cadence even during 1-minute RID retry), one-shot via
--collect-hii; docs/DATA_SOURCES.md catalogs all probed endpoints.
2026-08-11 14:44:53 +07:00
grabowski ab8a10dd75 Add install.sh and fix service unit placeholder
- scripts/install.sh: one-command hardened deploy (creates the water-monitor
  system user, deploys to /opt, builds a uv-managed venv, installs and enables
  the systemd unit). Idempotent; excludes .env/*.db/stations.json from sync so
  runtime state is preserved.
- Fix placeholder Documentation= URL in water-monitor.service.
- README: document the script as the primary systemd install path, with manual
  steps kept as a fallback.
2026-07-22 12:55:17 +07:00
grabowski 17a716fcd0 Version bump: 3.1.2 3.1.3 (Force new build)
Release - Northern Thailand Ping River Monitor / Create Release (push) Successful in 7s
Security & Dependency Updates / Dependency Security Scan (push) Successful in 35s
Security & Dependency Updates / Check for Dependency Updates (push) Has been cancelled
Security & Dependency Updates / Code Quality Metrics (push) Has been cancelled
Security & Dependency Updates / Security Summary (push) Has been cancelled
Security & Dependency Updates / License Compliance (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.11) (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.12) (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.9) (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Build Release Images (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Security Scan (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Deploy Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Validate Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Notify Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.10) (push) Has been cancelled
Version Updates:
- Core application: src/__init__.py, src/main.py, src/web_api.py
- Package configuration: setup.py
- Documentation: README.md, docs/GITEA_WORKFLOWS.md
- Workflows: .gitea/workflows/docs.yml, .gitea/workflows/release.yml
- Scripts: generate_badges.py, init_git scripts
- Tests: test_integration.py
- Deployment docs: GITEA_SETUP_SUMMARY.md, DEPLOYMENT_CHECKLIST.md

 Purpose:
- Force new build process after workflow fixes
- Test updated security.yml without YAML errors
- Verify setup.py robustness improvements
- Trigger clean CI/CD pipeline execution

 All version references synchronized at v3.1.3
 Ready for new build and deployment testing
2025-08-12 17:47:26 +07:00
grabowski 40aef686af Fix: Replace GitHub checkout with Gitea checkout + Version bump
Release - Northern Thailand Ping River Monitor / Create Release (push) Failing after 1s
Release - Northern Thailand Ping River Monitor / Build Release Images (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Test Release Build (3.10) (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Test Release Build (3.11) (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Test Release Build (3.12) (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Test Release Build (3.9) (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Security Scan (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Deploy Release (push) Has been skipped
Release - Northern Thailand Ping River Monitor / Validate Release (push) Has been skipped
Security & Dependency Updates / Dependency Security Scan (push) Failing after 1s
Security & Dependency Updates / Docker Security Scan (push) Failing after 1s
Security & Dependency Updates / License Compliance (push) Failing after 1s
Security & Dependency Updates / Check for Dependency Updates (push) Failing after 1s
Security & Dependency Updates / Code Quality Metrics (push) Failing after 1s
Release - Northern Thailand Ping River Monitor / Notify Release (push) Successful in 1s
Security & Dependency Updates / Security Summary (push) Failing after 3s
Checkout Action Migration:
- Replace all 'actions/checkout@v4' with 'https://gitea.com/actions/checkout'
- Fixes 'Bad credentials' errors when workflows try to access GitHub API
- Native Gitea checkout action eliminates authentication issues
- Applied across all 4 workflow files (CI, Security, Release, Docs)

 Version Increment: 3.1.1  3.1.2
- Core application version updates
- Web API version synchronization
- Documentation version alignment
- Badge and release example updates

 Problem Solved:
- Workflows no longer attempt GitHub API calls
- Gitea-native checkout action handles repository access properly
- Eliminates 'Retrieving the default branch name' failures
- Cleaner workflow execution without authentication errors

 Files Updated:
- 4 workflow files: checkout action replacement
- 13 files: version number updates
- Consistent v3.1.2 across all components

 Benefits:
- Workflows will now run successfully in Gitea
- No more GitHub API authentication failures
- Native Gitea action compatibility
- Ready for successful CI/CD pipeline execution
2025-08-12 17:06:20 +07:00
grabowski 19e182c53b Version bump: 3.1.0 3.1.1
Release - Northern Thailand Ping River Monitor / Create Release (push) Successful in 6s
Security & Dependency Updates / Dependency Security Scan (push) Successful in 19s
Security & Dependency Updates / Docker Security Scan (push) Successful in 1m12s
Security & Dependency Updates / License Compliance (push) Successful in 11s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 14s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 9s
Release - Northern Thailand Ping River Monitor / Test Release Build (3.10) (push) Failing after 1m17s
Release - Northern Thailand Ping River Monitor / Test Release Build (3.11) (push) Failing after 23s
Release - Northern Thailand Ping River Monitor / Test Release Build (3.9) (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Build Release Images (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Security Scan (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Deploy Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Validate Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Notify Release (push) Has been cancelled
Release - Northern Thailand Ping River Monitor / Test Release Build (3.12) (push) Has been cancelled
Security & Dependency Updates / Security Summary (push) Has been cancelled
Version Updates:
- Core application version (src/__init__.py)
- Web API version (src/web_api.py)
- Main application logging (src/main.py)
- Package setup version (setup.py)
- Documentation generation (docs workflow)
- Release workflow example version
- Badge generation script
- Integration test version display
- README.md badge version
- Setup and deployment documentation
- Git initialization scripts

 Patch Release (3.1.1):
- Workflow token migration fixes (GITHUB_TOKEN  GH_TOKEN)
- Pip installation warning elimination
- Improved workflow reliability and logging
- Better Gitea compatibility
- Enhanced error handling and validation

 Files Updated:
- 13 files with version references updated
- Consistent versioning across all components
- Ready for release tagging and deployment
2025-08-12 16:52:39 +07:00
grabowski af62cfef0b Initial commit: Northern Thailand Ping River Monitor v3.1.0
Security & Dependency Updates / Dependency Security Scan (push) Successful in 29s
Security & Dependency Updates / Docker Security Scan (push) Failing after 53s
Security & Dependency Updates / License Compliance (push) Successful in 13s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 19s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 11s
Security & Dependency Updates / Security Summary (push) Successful in 7s
Features:
- Real-time water level monitoring for Ping River Basin (16 stations)
- Coverage from Chiang Dao to Nakhon Sawan in Northern Thailand
- FastAPI web interface with interactive dashboard and station management
- Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics)
- Comprehensive monitoring with health checks and metrics collection
- Docker deployment with Grafana integration
- Production-ready architecture with enterprise-grade observability

 CI/CD & Automation:
- Complete Gitea Actions workflows for CI/CD, security, and releases
- Multi-Python version testing (3.9-3.12)
- Multi-architecture Docker builds (amd64, arm64)
- Daily security scanning and dependency monitoring
- Automated documentation generation
- Performance testing and validation

 Production Ready:
- Type safety with Pydantic models and comprehensive type hints
- Data validation layer with range checking and error handling
- Rate limiting and request tracking for API protection
- Enhanced logging with rotation, colors, and performance metrics
- Station management API for dynamic CRUD operations
- Comprehensive documentation and deployment guides

 Technical Stack:
- Python 3.9+ with FastAPI and Pydantic
- Multi-database architecture with adapter pattern
- Docker containerization with multi-stage builds
- Grafana dashboards for visualization
- Gitea Actions for CI/CD automation
- Enterprise monitoring and alerting

 Ready for deployment to B4L infrastructure!
2025-08-12 15:40:24 +07:00