13 Commits

Author	SHA1	Message	Date
grabowski	60e70c2192	Fix validator to handle null discharge values properly ... - Make discharge field optional in data validator - Remove discharge from required fields list - Add explicit null check for discharge before float conversion - Prevent "float() argument must be a string or a real number, not 'NoneType'" errors - Allow records with valid water levels but malformed/null discharge data This completes the malformed data handling fix by updating the validator to match the parser's new behavior of allowing null discharge values. Before: Validator rejected records with null discharge After: Validator accepts records with null discharge, validates only if present 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-28 18:55:20 +07:00
grabowski	cc5c4522b8	Fix malformed discharge data handling to preserve water level data ... - Change data parsing logic to make discharge data optional - Water level data is now saved even when discharge values are malformed (e.g., "***") - Handle malformed discharge values gracefully with null instead of skipping entire record - Add specific handling for "***" discharge values from API - Improve data completeness by not discarding valid water level measurements Before: Entire station record was skipped if discharge was malformed After: Water level data is preserved, discharge set to null for malformed values Example fix: - wlvalues8: 1.6 (valid) + qvalues8: "***" (malformed) - Before: No record saved - After: Record saved with water_level=1.6, discharge=null 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-28 18:41:39 +07:00
grabowski	6846091522	Implement smart date selection for data fetching ... - Add intelligent date selection based on current time - Before 01:00: fetch yesterday's data only (API not updated yet) - After 01:00: try today's data first, fallback to yesterday if needed - Improve data availability by adapting to API update patterns - Add comprehensive logging for date selection decisions This ensures optimal data fetching regardless of the time of day: - Early morning (00:00-00:59): fetches yesterday (reliable) - Rest of day (01:00-23:59): tries today first, falls back to yesterday 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-28 18:30:53 +07:00
grabowski	4cc792157f	Implement adaptive scheduler with intelligent retry logic ... - Replace fixed hourly schedule with adaptive scheduling system - Switch to 1-minute retries when no data is available from API - Return to hourly schedule once data is successfully fetched - Fix data fetching to use yesterday's date (API has 1-day delay) - Add comprehensive logging for scheduler mode changes - Improve resilience against API data availability issues The scheduler now intelligently adapts to data availability: - Normal mode: hourly runs at top of each hour - Retry mode: minute-based retries until data is available - Automatic mode switching based on fetch success/failure 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-28 18:24:33 +07:00
grabowski	0ff58ecb13	Add historical data import functionality ... - Add import_historical_data() method to EnhancedWaterMonitorScraper - Support date range imports with Buddhist calendar API format - Add CLI arguments --import-historical and --force-overwrite - Include API rate limiting and skip existing data option - Enable importing years of historical water level data 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-28 14:46:54 +07:00
grabowski	bd812ca5ca	Improve scheduler to run immediately then wait for next full hour ... - Run initial data collection immediately on startup - Calculate wait time to next full hour (e.g., 22:12 start waits until 23:00) - Schedule subsequent runs at top of each hour (:00 minutes) - Display next scheduled run time to user for better visibility 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-26 22:51:00 +07:00
grabowski	ca730e484b	Add comprehensive Matrix alerting system with Grafana integration ... - Implement custom Python alerting system (src/alerting.py) with water level monitoring, data freshness checks, and Matrix notifications - Add complete Grafana Matrix alerting setup guide (docs/GRAFANA_MATRIX_SETUP.md) with webhook configuration, alert rules, and notification policies - Create Matrix quick start guide (docs/MATRIX_QUICK_START.md) for rapid deployment - Integrate alerting commands into main application (--alert-check, --alert-test) - Add Matrix configuration to environment variables (.env.example) - Update Makefile with alerting targets (alert-check, alert-test) - Enhance status command to show Matrix notification status - Support station-specific water level thresholds and escalation rules - Provide dual alerting approach: native Grafana alerts and custom Python system 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-26 16:18:02 +07:00
grabowski	6c7c128b4d	Major refactor: Migrate to uv, add PostgreSQL support, and comprehensive tooling ... - **Migration to uv package manager**: Replace pip/requirements with modern pyproject.toml - Add pyproject.toml with complete dependency management - Update all scripts and Makefile to use uv commands - Maintain backward compatibility with existing workflows - **PostgreSQL integration and migration tools**: - Enhanced config.py with automatic password URL encoding - Complete PostgreSQL setup scripts and documentation - High-performance SQLite to PostgreSQL migration tool (91x speed improvement) - Support for both connection strings and individual components - **Executable distribution system**: - PyInstaller integration for standalone .exe creation - Automated build scripts with batch file generation - Complete packaging system for end-user distribution - **Enhanced data management**: - Fix --fill-gaps command with proper method implementation - Add gap detection and historical data backfill capabilities - Implement data update functionality for existing records - Add comprehensive database adapter methods - **Developer experience improvements**: - Password encoding tools for special characters - Interactive setup wizards for PostgreSQL configuration - Comprehensive documentation and migration guides - Automated testing and validation tools 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-09-26 15:10:10 +07:00
grabowski	730cbac7ae	fixed time import	2025-08-14 10:49:31 +07:00
grabowski	17a716fcd0	Version bump: 3.1.2 3.1.3 (Force new build) ... Version Updates: - Core application: src/__init__.py, src/main.py, src/web_api.py - Package configuration: setup.py - Documentation: README.md, docs/GITEA_WORKFLOWS.md - Workflows: .gitea/workflows/docs.yml, .gitea/workflows/release.yml - Scripts: generate_badges.py, init_git scripts - Tests: test_integration.py - Deployment docs: GITEA_SETUP_SUMMARY.md, DEPLOYMENT_CHECKLIST.md Purpose: - Force new build process after workflow fixes - Test updated security.yml without YAML errors - Verify setup.py robustness improvements - Trigger clean CI/CD pipeline execution All version references synchronized at v3.1.3 Ready for new build and deployment testing	2025-08-12 17:47:26 +07:00
grabowski	40aef686af	Fix: Replace GitHub checkout with Gitea checkout + Version bump ... Checkout Action Migration: - Replace all 'actions/checkout@v4' with 'https://gitea.com/actions/checkout' - Fixes 'Bad credentials' errors when workflows try to access GitHub API - Native Gitea checkout action eliminates authentication issues - Applied across all 4 workflow files (CI, Security, Release, Docs) Version Increment: 3.1.1 3.1.2 - Core application version updates - Web API version synchronization - Documentation version alignment - Badge and release example updates Problem Solved: - Workflows no longer attempt GitHub API calls - Gitea-native checkout action handles repository access properly - Eliminates 'Retrieving the default branch name' failures - Cleaner workflow execution without authentication errors Files Updated: - 4 workflow files: checkout action replacement - 13 files: version number updates - Consistent v3.1.2 across all components Benefits: - Workflows will now run successfully in Gitea - No more GitHub API authentication failures - Native Gitea action compatibility - Ready for successful CI/CD pipeline execution	2025-08-12 17:06:20 +07:00
grabowski	19e182c53b	Version bump: 3.1.0 3.1.1 ... Version Updates: - Core application version (src/__init__.py) - Web API version (src/web_api.py) - Main application logging (src/main.py) - Package setup version (setup.py) - Documentation generation (docs workflow) - Release workflow example version - Badge generation script - Integration test version display - README.md badge version - Setup and deployment documentation - Git initialization scripts Patch Release (3.1.1): - Workflow token migration fixes (GITHUB_TOKEN GH_TOKEN) - Pip installation warning elimination - Improved workflow reliability and logging - Better Gitea compatibility - Enhanced error handling and validation Files Updated: - 13 files with version references updated - Consistent versioning across all components - Ready for release tagging and deployment	2025-08-12 16:52:39 +07:00
grabowski	af62cfef0b	Initial commit: Northern Thailand Ping River Monitor v3.1.0 ... Features: - Real-time water level monitoring for Ping River Basin (16 stations) - Coverage from Chiang Dao to Nakhon Sawan in Northern Thailand - FastAPI web interface with interactive dashboard and station management - Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics) - Comprehensive monitoring with health checks and metrics collection - Docker deployment with Grafana integration - Production-ready architecture with enterprise-grade observability CI/CD & Automation: - Complete Gitea Actions workflows for CI/CD, security, and releases - Multi-Python version testing (3.9-3.12) - Multi-architecture Docker builds (amd64, arm64) - Daily security scanning and dependency monitoring - Automated documentation generation - Performance testing and validation Production Ready: - Type safety with Pydantic models and comprehensive type hints - Data validation layer with range checking and error handling - Rate limiting and request tracking for API protection - Enhanced logging with rotation, colors, and performance metrics - Station management API for dynamic CRUD operations - Comprehensive documentation and deployment guides Technical Stack: - Python 3.9+ with FastAPI and Pydantic - Multi-database architecture with adapter pattern - Docker containerization with multi-stage builds - Grafana dashboards for visualization - Gitea Actions for CI/CD automation - Enterprise monitoring and alerting Ready for deployment to B4L infrastructure!	2025-08-12 15:40:24 +07:00