14 Commits

Include renames

Author	SHA1	Message	Date
grabowski	1ec5cfb4df	perf: gzip responses; multi-worker serving with single collection leader ... GZipMiddleware (min 500 bytes) compresses the dashboard HTML ~4x and station JSON up to ~100x, end-to-end through the Caddy TLS terminator — production load testing showed the deployment is bandwidth-bound once the response caches hit, so compression is the capacity lever. WEB_WORKERS (default 2) runs uvicorn multi-process via the app import string. Every worker executes the lifespan, so a localhost lock port (COLLECTION_LEADER_PORT, default 8901) elects exactly one background-collection leader per machine — RID/HII polling stays once-per-cycle instead of once-per-worker; the lock releases with the process. Locust clients now send Accept-Encoding so future runs measure compressed transfer, as browsers do.	2026-08-12 11:34:12 +07:00
grabowski	1845ef7203	feat: hourly HII/ThaiWater rainfall + backup water-level collection ... Poll the open api-v3.thaiwater.net public endpoints (rain_24h, waterlevel_load), filter to the Ping basin, and persist to new hii_rain_stations/hii_rainfall and hii_wl_stations/hii_waterlevel tables (auto-created; sqlite/postgresql/mysql). Water levels stay in their own tables since HII reports m MSL from a different station set; rid_code maps mirrors like ridhydro_P.1 to P.1 and offset_msl converts MSL to gauge datum. Runs every scraping cycle in web-api and continuous modes (hourly cadence even during 1-minute RID retry), one-shot via --collect-hii; docs/DATA_SOURCES.md catalogs all probed endpoints.	2026-08-11 14:44:53 +07:00
grabowski	5e62ea529d	feat: --fill-gaps all repairs missing data across the whole DB range ... Gap detection is now hour-granular: days with partial data (missing hourly slots) are re-fetched, not just days with no rows at all. The API's hour-24-is-next-midnight quirk is handled by re-fetching day D-1 when day D is missing its 00:00 slot. SQL adapters gain single-query range and recorded-hours lookups; non-SQL backends fall back to the old day-granular check.	2026-08-10 22:13:58 +07:00
grabowski	9cac9c4d2a	style: apply black/isort across the repo; make CI mypy advisory ... The push-CI gates (black/isort/mypy) had never actually run before the branch-trigger fix, and the codebase predates them. Formatting is now black/isort clean repo-wide. mypy keeps running but non-blocking: 86 pre-existing errors are a separate cleanup, not a gate to hold hostage.	2026-08-10 15:57:00 +07:00
grabowskiandClaude	58cc60ba19	Integrate automatic alerting into continuous monitoring ... - Alerts now run automatically after every successful new data fetch - Works for both hourly fetches and retry mode exits - Alert check runs when fresh data is saved to database - Logs alert results (total generated and sent count) Co-Authored-By: Claude <noreply@anthropic.com>	2025-10-03 16:44:57 +07:00
grabowskiandClaude	4cc792157f	Implement adaptive scheduler with intelligent retry logic ... - Replace fixed hourly schedule with adaptive scheduling system - Switch to 1-minute retries when no data is available from API - Return to hourly schedule once data is successfully fetched - Fix data fetching to use yesterday's date (API has 1-day delay) - Add comprehensive logging for scheduler mode changes - Improve resilience against API data availability issues The scheduler now intelligently adapts to data availability: - Normal mode: hourly runs at top of each hour - Retry mode: minute-based retries until data is available - Automatic mode switching based on fetch success/failure 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-28 18:24:33 +07:00
grabowskiandClaude	0ff58ecb13	Add historical data import functionality ... - Add import_historical_data() method to EnhancedWaterMonitorScraper - Support date range imports with Buddhist calendar API format - Add CLI arguments --import-historical and --force-overwrite - Include API rate limiting and skip existing data option - Enable importing years of historical water level data 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-28 14:46:54 +07:00
grabowskiandClaude	bd812ca5ca	Improve scheduler to run immediately then wait for next full hour ... - Run initial data collection immediately on startup - Calculate wait time to next full hour (e.g., 22:12 start waits until 23:00) - Schedule subsequent runs at top of each hour (:00 minutes) - Display next scheduled run time to user for better visibility 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-26 22:51:00 +07:00
grabowskiandClaude	ca730e484b	Add comprehensive Matrix alerting system with Grafana integration ... - Implement custom Python alerting system (src/alerting.py) with water level monitoring, data freshness checks, and Matrix notifications - Add complete Grafana Matrix alerting setup guide (docs/GRAFANA_MATRIX_SETUP.md) with webhook configuration, alert rules, and notification policies - Create Matrix quick start guide (docs/MATRIX_QUICK_START.md) for rapid deployment - Integrate alerting commands into main application (--alert-check, --alert-test) - Add Matrix configuration to environment variables (.env.example) - Update Makefile with alerting targets (alert-check, alert-test) - Enhance status command to show Matrix notification status - Support station-specific water level thresholds and escalation rules - Provide dual alerting approach: native Grafana alerts and custom Python system 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-26 16:18:02 +07:00
grabowski	730cbac7ae	fixed time import	2025-08-14 10:49:31 +07:00
grabowski	17a716fcd0	Version bump: 3.1.2 3.1.3 (Force new build) ... Version Updates: - Core application: src/__init__.py, src/main.py, src/web_api.py - Package configuration: setup.py - Documentation: README.md, docs/GITEA_WORKFLOWS.md - Workflows: .gitea/workflows/docs.yml, .gitea/workflows/release.yml - Scripts: generate_badges.py, init_git scripts - Tests: test_integration.py - Deployment docs: GITEA_SETUP_SUMMARY.md, DEPLOYMENT_CHECKLIST.md Purpose: - Force new build process after workflow fixes - Test updated security.yml without YAML errors - Verify setup.py robustness improvements - Trigger clean CI/CD pipeline execution All version references synchronized at v3.1.3 Ready for new build and deployment testing	2025-08-12 17:47:26 +07:00
grabowski	40aef686af	Fix: Replace GitHub checkout with Gitea checkout + Version bump ... Checkout Action Migration: - Replace all 'actions/checkout@v4' with 'https://gitea.com/actions/checkout' - Fixes 'Bad credentials' errors when workflows try to access GitHub API - Native Gitea checkout action eliminates authentication issues - Applied across all 4 workflow files (CI, Security, Release, Docs) Version Increment: 3.1.1 3.1.2 - Core application version updates - Web API version synchronization - Documentation version alignment - Badge and release example updates Problem Solved: - Workflows no longer attempt GitHub API calls - Gitea-native checkout action handles repository access properly - Eliminates 'Retrieving the default branch name' failures - Cleaner workflow execution without authentication errors Files Updated: - 4 workflow files: checkout action replacement - 13 files: version number updates - Consistent v3.1.2 across all components Benefits: - Workflows will now run successfully in Gitea - No more GitHub API authentication failures - Native Gitea action compatibility - Ready for successful CI/CD pipeline execution	2025-08-12 17:06:20 +07:00
grabowski	19e182c53b	Version bump: 3.1.0 3.1.1 ... Version Updates: - Core application version (src/__init__.py) - Web API version (src/web_api.py) - Main application logging (src/main.py) - Package setup version (setup.py) - Documentation generation (docs workflow) - Release workflow example version - Badge generation script - Integration test version display - README.md badge version - Setup and deployment documentation - Git initialization scripts Patch Release (3.1.1): - Workflow token migration fixes (GITHUB_TOKEN GH_TOKEN) - Pip installation warning elimination - Improved workflow reliability and logging - Better Gitea compatibility - Enhanced error handling and validation Files Updated: - 13 files with version references updated - Consistent versioning across all components - Ready for release tagging and deployment	2025-08-12 16:52:39 +07:00
grabowski	af62cfef0b	Initial commit: Northern Thailand Ping River Monitor v3.1.0 ... Features: - Real-time water level monitoring for Ping River Basin (16 stations) - Coverage from Chiang Dao to Nakhon Sawan in Northern Thailand - FastAPI web interface with interactive dashboard and station management - Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics) - Comprehensive monitoring with health checks and metrics collection - Docker deployment with Grafana integration - Production-ready architecture with enterprise-grade observability CI/CD & Automation: - Complete Gitea Actions workflows for CI/CD, security, and releases - Multi-Python version testing (3.9-3.12) - Multi-architecture Docker builds (amd64, arm64) - Daily security scanning and dependency monitoring - Automated documentation generation - Performance testing and validation Production Ready: - Type safety with Pydantic models and comprehensive type hints - Data validation layer with range checking and error handling - Rate limiting and request tracking for API protection - Enhanced logging with rotation, colors, and performance metrics - Station management API for dynamic CRUD operations - Comprehensive documentation and deployment guides Technical Stack: - Python 3.9+ with FastAPI and Pydantic - Multi-database architecture with adapter pattern - Docker containerization with multi-stage builds - Grafana dashboards for visualization - Gitea Actions for CI/CD automation - Enterprise monitoring and alerting Ready for deployment to B4L infrastructure!	2025-08-12 15:40:24 +07:00