chore: declutter the repo

Removes 26 tracked files that no longer describe or serve the running
system, verified one by one against the whole repo (source, tests, docs,
README, Makefile, Dockerfile, .gitea workflows, pyproject, packaging spec)
plus dynamic-reference paths, before deletion.

Root (11): one-off launch/setup write-ups from the project's first weeks
that document events which never happened the way they describe — a
github.com publication (the remote is self-hosted Gitea) and a 15-minute
scheduler (the service runs hourly). Also .gitlab-ci.yml (unused, CI is
.gitea/), .env.postgres and setup.py.backup (a placeholder env file and a
backup in version control), and the PyInstaller packaging trio
build_executable.py / build_simple.py / ping-river-monitor.spec, which
bundled docs that no longer exist and is not how this deploys.

docs (9): stale guides superseded by DATABASE_DEPLOYMENT_GUIDE,
GITEA_WORKFLOWS, FLOOD_FORECASTING and DATA_SOURCES, plus two snapshots
(PROJECT_STATUS, PROJECT_STRUCTURE) describing a 4-file src/ that is now 39.

scripts (5): one-shot bootstrap tools already run — init_git.sh/.bat,
generate_badges.py, migrate_geolocation.py, encode_password.py.

Every inbound reference was fixed rather than left dangling: README doc
index and migration section, three Makefile targets, the docs.yml summary
step, and the GITEA_WORKFLOWS resource list.

src/ is deliberately untouched. The audit proposed removing several live
modules; verification showed those proposals were mis-scoped and would
have broken production.

.gitignore now covers the agent tooling dirs, model eval output and
editor/shell leftovers — the working tree had collected 56 zero-byte
files named after fragments of shell commands.

136 tests pass; production modules import clean.

docs/ENHANCED_SCHEDULER_GUIDE.md

@@ -1,293 +0,0 @@
-# Enhanced Scheduler Guide
-
-This guide explains the new 15-minute scheduling system that runs continuously throughout each hour to ensure comprehensive data coverage.
-
-## ✅ **New Scheduling Behavior**
-
-### **15-Minute Schedule Pattern**
-- **Timing**: Runs every 15 minutes: 1:00, 1:15, 1:30, 1:45, 2:00, 2:15, 2:30, 2:45, etc.
-- **Hourly Full Checks**: At :00 minutes (includes gap filling and data updates)
-- **Quarter-Hour Quick Checks**: At :15, :30, :45 minutes (data fetch only)
-- **Continuous Coverage**: Ensures no data is missed throughout each hour
-
-### **Operation Types**
-- **Full Operations** (at :00): Data fetching + gap filling + data updates
-- **Quick Operations** (at :15, :30, :45): Data fetching only for performance
-
-## 🔧 **Technical Implementation**
-
-### **Scheduler States**
-```python
-# State tracking variables
-self.last_successful_update = None  # Timestamp of last successful data update
-self.retry_mode = False            # Whether in quick check mode (skip gap filling)
-self.next_hourly_check = None      # Next scheduled hourly check
-```
-
-### **Quarter-Hour Check Process**
-```python
-def quarter_hour_check(self):
-    """15-minute check for new data"""
-    current_time = datetime.datetime.now()
-    minute = current_time.minute
-    
-    # Determine if this is a full hourly check (at :00) or a quarter-hour check
-    if minute == 0:
-        logging.info("=== HOURLY CHECK (00:00) ===")
-        self.retry_mode = False  # Full check with gap filling and updates
-    else:
-        logging.info(f"=== 15-MINUTE CHECK ({minute:02d}:00) ===")
-        self.retry_mode = True   # Skip gap filling and updates on 15-min checks
-    
-    new_data_found = self.run_scraping_cycle()
-    
-    if new_data_found:
-        self.last_successful_update = datetime.datetime.now()
-        if minute == 0:
-            logging.info("New data found during hourly check")
-        else:
-            logging.info(f"New data found during 15-minute check at :{minute:02d}")
-    else:
-        if minute == 0:
-            logging.info("No new data found during hourly check")
-        else:
-            logging.info(f"No new data found during 15-minute check at :{minute:02d}")
-```
-
-### **Scheduler Setup**
-```python
-def start_scheduler(self):
-    """Start enhanced scheduler with 15-minute checks"""
-    # Schedule checks every 15 minutes (at :00, :15, :30, :45)
-    schedule.every().hour.at(":00").do(self.quarter_hour_check)
-    schedule.every().hour.at(":15").do(self.quarter_hour_check)
-    schedule.every().hour.at(":30").do(self.quarter_hour_check)
-    schedule.every().hour.at(":45").do(self.quarter_hour_check)
-    
-    while True:
-        schedule.run_pending()
-        time.sleep(30)  # Check every 30 seconds
-```
-
-## 📊 **New Data Detection Logic**
-
-### **Smart Detection Algorithm**
-```python
-def has_new_data(self) -> bool:
-    """Check if there is new data available since last successful update"""
-    # Get most recent timestamp from database
-    latest_data = self.get_latest_data(limit=1)
-    
-    # Check if we should have newer data by now
-    now = datetime.datetime.now()
-    expected_latest = now.replace(minute=0, second=0, microsecond=0)
-    
-    # If current time is past 5 minutes after the hour, we should have data
-    if now.minute >= 5:
-        if latest_timestamp < expected_latest:
-            return True  # New data expected
-    
-    # Check if we have data for the previous hour
-    previous_hour = expected_latest - datetime.timedelta(hours=1)
-    if latest_timestamp < previous_hour:
-        return True  # Missing recent data
-    
-    return False  # Data is up to date
-```
-
-### **Actual Data Verification**
-```python
-# Compare timestamps before and after scraping
-initial_timestamp = get_latest_timestamp_before_scraping()
-# ... perform scraping ...
-latest_timestamp = get_latest_timestamp_after_scraping()
-
-if initial_timestamp is None or latest_timestamp > initial_timestamp:
-    new_data_found = True
-    self.last_successful_update = datetime.datetime.now()
-```
-
-## 🚀 **Operational Modes**
-
-### **Mode 1: Full Hourly Operation (at :00)**
-- **Schedule**: Every hour at :00 minutes (1:00, 2:00, 3:00, etc.)
-- **Operations**: 
-  - ✅ Fetch current data
-  - ✅ Fill data gaps (last 7 days)
-  - ✅ Update existing data (last 2 days)
-- **Purpose**: Comprehensive data collection and maintenance
-
-### **Mode 2: Quick 15-Minute Checks (at :15, :30, :45)**
-- **Schedule**: Every 15 minutes at quarter-hour marks
-- **Operations**:
-  - ✅ Fetch current data only
-  - ❌ Skip gap filling (performance optimization)
-  - ❌ Skip data updates (performance optimization)
-- **Purpose**: Ensure no new data is missed between hourly checks
-
-## 📋 **Logging Output Examples**
-
-### **Successful Hourly Check (at :00)**
-```
-2025-07-26 01:00:00,123 - INFO - === HOURLY CHECK (00:00) ===
-2025-07-26 01:00:00,124 - INFO - Starting scraping cycle...
-2025-07-26 01:00:01,456 - INFO - Successfully fetched 384 data points from API
-2025-07-26 01:00:02,789 - INFO - New data found: 2025-07-26 01:00:00
-2025-07-26 01:00:03,012 - INFO - Filled 5 data gaps
-2025-07-26 01:00:04,234 - INFO - Updated 2 existing measurements
-2025-07-26 01:00:04,235 - INFO - New data found during hourly check
-```
-
-### **15-Minute Quick Check (at :15, :30, :45)**
-```
-2025-07-26 01:15:00,123 - INFO - === 15-MINUTE CHECK (15:00) ===
-2025-07-26 01:15:00,124 - INFO - Starting scraping cycle...
-2025-07-26 01:15:01,456 - INFO - Successfully fetched 299 data points from API
-2025-07-26 01:15:02,789 - INFO - New data found: 2025-07-26 01:00:00
-2025-07-26 01:15:02,790 - INFO - New data found during 15-minute check at :15
-```
-
-### **Continuous 15-Minute Pattern**
-```
-2025-07-26 01:00:00,123 - INFO - === HOURLY CHECK (00:00) ===
-2025-07-26 01:00:04,235 - INFO - New data found during hourly check
-
-2025-07-26 01:15:00,123 - INFO - === 15-MINUTE CHECK (15:00) ===
-2025-07-26 01:15:02,790 - INFO - No new data found during 15-minute check at :15
-
-2025-07-26 01:30:00,123 - INFO - === 15-MINUTE CHECK (30:00) ===
-2025-07-26 01:30:02,790 - INFO - No new data found during 15-minute check at :30
-
-2025-07-26 01:45:00,123 - INFO - === 15-MINUTE CHECK (45:00) ===
-2025-07-26 01:45:02,790 - INFO - No new data found during 15-minute check at :45
-
-2025-07-26 02:00:00,123 - INFO - === HOURLY CHECK (00:00) ===
-2025-07-26 02:00:04,235 - INFO - New data found during hourly check
-```
-
-## ⚙️ **Configuration Options**
-
-### **Environment Variables**
-```bash
-# Retry interval (default: 5 minutes)
-export RETRY_INTERVAL_MINUTES=5
-
-# Data availability buffer (default: 5 minutes after hour)
-export DATA_BUFFER_MINUTES=5
-
-# Gap filling days (default: 7 days)
-export GAP_FILL_DAYS=7
-
-# Update check days (default: 2 days)
-export UPDATE_DAYS=2
-```
-
-### **Scheduler Timing**
-```python
-# Hourly checks at top of hour
-schedule.every().hour.at(":00").do(self.hourly_check)
-
-# 5-minute retries (dynamically scheduled)
-schedule.every(5).minutes.do(self.retry_check).tag('retry')
-
-# Check every 30 seconds for responsive retry scheduling
-time.sleep(30)
-```
-
-## 🔍 **Performance Optimizations**
-
-### **Retry Mode Optimizations**
-- **Skip Gap Filling**: Avoids expensive historical data fetching during retries
-- **Skip Data Updates**: Avoids comparison operations during retries
-- **Focused API Calls**: Only fetches current day data during retries
-- **Reduced Database Queries**: Minimal database operations during retries
-
-### **Resource Management**
-- **API Rate Limiting**: 1-second delays between API calls
-- **Database Connection Pooling**: Efficient connection reuse
-- **Memory Efficiency**: Selective data processing
-- **Error Recovery**: Automatic retry with exponential backoff
-
-## 🛠️ **Troubleshooting**
-
-### **Common Scenarios**
-
-#### **Stuck in Retry Mode**
-```
-# Check if API is returning data
-curl -X POST https://hyd-app-db.rid.go.th/webservice/getGroupHourlyWaterLevelReportAllHL.ashx
-
-# Check database connectivity
-python water_scraper_v3.py --check-gaps 1
-
-# Manual data fetch test
-python water_scraper_v3.py --test
-```
-
-#### **Missing Hourly Triggers**
-```
-# Check system time synchronization
-timedatectl status
-
-# Verify scheduler is running
-ps aux | grep water_scraper
-
-# Check logs for scheduler activity
-tail -f water_monitor.log | grep "HOURLY CHECK"
-```
-
-#### **False New Data Detection**
-```
-# Check latest data in database
-sqlite3 water_monitoring.db "SELECT MAX(timestamp) FROM water_measurements;"
-
-# Verify timestamp parsing
-python -c "
-import datetime
-print('Current hour:', datetime.datetime.now().replace(minute=0, second=0, microsecond=0))
-"
-```
-
-## 📈 **Monitoring and Alerts**
-
-### **Key Metrics to Monitor**
-- **Hourly Success Rate**: Percentage of hourly checks that find new data
-- **Retry Duration**: How long system stays in retry mode
-- **Data Freshness**: Time since last successful data update
-- **API Response Time**: Performance of data fetching operations
-
-### **Alert Conditions**
-- **Extended Retry Mode**: System in retry mode for > 30 minutes
-- **No Data for 2+ Hours**: No new data found for extended period
-- **High Error Rate**: Multiple consecutive API failures
-- **Database Issues**: Connection or save failures
-
-### **Health Check Script**
-```bash
-#!/bin/bash
-# Check if system is stuck in retry mode
-RETRY_COUNT=$(tail -n 100 water_monitor.log | grep -c "RETRY CHECK")
-if [ $RETRY_COUNT -gt 6 ]; then
-    echo "WARNING: System may be stuck in retry mode ($RETRY_COUNT retries in last 100 log entries)"
-fi
-
-# Check data freshness
-LATEST_DATA=$(sqlite3 water_monitoring.db "SELECT MAX(timestamp) FROM water_measurements;")
-echo "Latest data timestamp: $LATEST_DATA"
-```
-
-## 🎯 **Best Practices**
-
-### **Production Deployment**
-1. **Monitor Logs**: Watch for retry mode patterns
-2. **Set Alerts**: Configure notifications for extended retry periods
-3. **Regular Maintenance**: Weekly gap filling and data validation
-4. **Backup Strategy**: Regular database backups before major operations
-
-### **Performance Tuning**
-1. **Adjust Buffer Time**: Modify data availability buffer based on API patterns
-2. **Optimize Retry Interval**: Balance between responsiveness and API load
-3. **Database Indexing**: Ensure proper indexes for timestamp queries
-4. **Connection Pooling**: Configure appropriate database connection limits
-
-This enhanced scheduler ensures reliable, efficient, and intelligent water level monitoring with automatic adaptation to data availability patterns.

docs/ENHANCEMENT_SUMMARY.md

@@ -1,227 +0,0 @@
-# 🚀 Northern Thailand Ping River Monitor - Enhancement Summary
-
-## 🎯 **What We've Accomplished**
-
-We've successfully transformed your water monitoring system from a simple scraper into a **production-ready, enterprise-grade monitoring platform** focused on the Ping River Basin in Northern Thailand, with modern web interfaces, station management capabilities, and comprehensive observability.
-
-## 🌟 **Major New Features Added**
-
-### 1. **FastAPI Web Interface** 🌐
-- **Interactive Dashboard** at `http://localhost:8000`
-- **REST API** with comprehensive endpoints
-- **Station Management** - Add, update, delete monitoring stations
-- **Real-time Health Monitoring** 
-- **Manual Data Collection Triggers**
-- **Interactive API Documentation** at `/docs`
-- **CORS Support** for web applications
-
-### 2. **Enhanced Architecture** 🏗️
-- **Type Safety** with Pydantic models and comprehensive type hints
-- **Data Validation Layer** with range checking and error handling
-- **Custom Exception Classes** for better error management
-- **Modular Design** with separated concerns
-
-### 3. **Observability & Monitoring** 📊
-- **Metrics Collection System** (counters, gauges, histograms)
-- **Health Checks** for database, API, and system resources
-- **Performance Tracking** with response times and success rates
-- **Enhanced Logging** with colors, rotation, and performance logs
-
-### 4. **Production Features** 🚀
-- **Rate Limiting** to prevent API abuse
-- **Request Tracking** with detailed statistics
-- **Configuration Validation** on startup
-- **Graceful Error Handling** and recovery
-- **Background Task Management**
-
-## 📁 **New Files Created**
-
-```
-src/
-├── models.py              # Data models and type definitions
-├── exceptions.py          # Custom exception classes
-├── validators.py          # Data validation layer
-├── metrics.py            # Metrics collection system
-├── health_check.py       # Health monitoring system
-├── rate_limiter.py       # Rate limiting and request tracking
-├── logging_config.py     # Enhanced logging configuration
-├── web_api.py           # FastAPI web interface
-├── main.py              # Enhanced CLI with multiple modes
-└── __init__.py          # Package initialization
-
-# Root files
-├── run.py                # Simple startup script
-├── test_integration.py   # Integration test suite
-├── test_api.py          # API endpoint tests
-└── ENHANCEMENT_SUMMARY.md # This file
-```
-
-## 🔧 **Enhanced Existing Files**
-
-- **`src/water_scraper_v3.py`** - Integrated new features, metrics, validation
-- **`src/config.py`** - Added configuration validation
-- **`requirements.txt`** - Added FastAPI, Pydantic, and monitoring dependencies
-- **`docker-compose.victoriametrics.yml`** - Added web API service
-- **`Dockerfile`** - Updated for new startup script
-- **`README.md`** - Updated with new features and usage instructions
-
-## 🌐 **Web API Endpoints**
-
-| Endpoint | Method | Description |
-|----------|--------|-------------|
-| `/` | GET | Interactive dashboard |
-| `/docs` | GET | API documentation |
-| `/health` | GET | System health status |
-| `/metrics` | GET | Application metrics |
-| `/stations` | GET | List all monitoring stations |
-| `/measurements/latest` | GET | Latest measurements |
-| `/measurements/station/{code}` | GET | Station-specific data |
-| `/scrape/trigger` | POST | Trigger manual data collection |
-| `/scraping/status` | GET | Scraping status and statistics |
-| `/config` | GET | Current configuration (masked) |
-
-## 🚀 **Usage Examples**
-
-### **Traditional Mode (Enhanced)**
-```bash
-# Test single cycle
-python run.py --test
-
-# Continuous monitoring
-python run.py
-
-# Fill data gaps
-python run.py --fill-gaps 7
-
-# Show system status
-python run.py --status
-```
-
-### **Web API Mode (NEW!)**
-```bash
-# Start web API server
-python run.py --web-api
-
-# Access dashboard
-open http://localhost:8000
-
-# View API documentation
-open http://localhost:8000/docs
-```
-
-### **Docker Deployment**
-```bash
-# Start complete stack
-docker-compose -f docker-compose.victoriametrics.yml up -d
-
-# Services available:
-# - Water API: http://localhost:8000
-# - Grafana: http://localhost:3000
-# - VictoriaMetrics: http://localhost:8428
-```
-
-## 📊 **Monitoring & Observability**
-
-### **Built-in Metrics**
-- API request counts and response times
-- Database connection status and save operations
-- Scraping cycle success/failure rates
-- System resource usage (memory, etc.)
-
-### **Health Checks**
-- Database connectivity and data freshness
-- External API availability
-- Memory usage monitoring
-- Overall system health status
-
-### **Enhanced Logging**
-- Colored console output for better readability
-- File rotation to prevent disk space issues
-- Performance logging for optimization
-- Structured logging with proper levels
-
-## 🔒 **Production Ready Features**
-
-### **Security & Reliability**
-- Rate limiting to prevent API abuse
-- Input validation and sanitization
-- Graceful error handling and recovery
-- Configuration validation on startup
-
-### **Performance**
-- Efficient metrics collection with minimal overhead
-- Background task management
-- Connection pooling and resource management
-- Optimized database operations
-
-### **Scalability**
-- Modular architecture for easy extension
-- Async support for high concurrency
-- Configurable resource limits
-- Health checks for load balancer integration
-
-## 🧪 **Testing**
-
-### **Integration Tests**
-```bash
-# Run all integration tests
-python test_integration.py
-```
-
-### **API Tests**
-```bash
-# Test API endpoints (server must be running)
-python test_api.py
-```
-
-## 📈 **Performance Improvements**
-
-1. **Request Tracking** - Monitor API performance and success rates
-2. **Rate Limiting** - Prevent API abuse and ensure stability
-3. **Data Validation** - Catch errors early and improve data quality
-4. **Metrics Collection** - Identify bottlenecks and optimization opportunities
-5. **Health Monitoring** - Proactive issue detection and alerting
-
-## 🎉 **Benefits Achieved**
-
-### **For Developers**
-- **Better Developer Experience** with type hints and validation
-- **Easier Debugging** with enhanced logging and error messages
-- **Comprehensive Testing** with integration and API tests
-- **Modern Architecture** following best practices
-
-### **For Operations**
-- **Web Dashboard** for easy monitoring and management
-- **Health Checks** for automated monitoring integration
-- **Metrics Collection** for performance analysis
-- **Production-Ready** deployment with Docker support
-
-### **For Users**
-- **REST API** for integration with other systems
-- **Real-time Data Access** via web interface
-- **Manual Controls** for triggering data collection
-- **Status Monitoring** for system visibility
-
-## 🔮 **Future Enhancement Opportunities**
-
-1. **Authentication & Authorization** - Add user management and API keys
-2. **Real-time WebSocket Updates** - Live data streaming to web clients
-3. **Advanced Analytics** - Trend analysis and forecasting
-4. **Alert System** - Email/SMS notifications for critical conditions
-5. **Multi-tenant Support** - Support for multiple organizations
-6. **Data Export** - CSV, Excel, and other format exports
-7. **Mobile App** - React Native or Flutter mobile interface
-
-## 🏆 **Summary**
-
-Your Thailand Water Monitor has been transformed from a simple data scraper into a **comprehensive, enterprise-grade monitoring platform** that includes:
-
-- ✅ **Modern Web Interface** with FastAPI
-- ✅ **Production-Ready Architecture** with proper error handling
-- ✅ **Comprehensive Monitoring** with metrics and health checks
-- ✅ **Type Safety** and data validation
-- ✅ **Enhanced Logging** and observability
-- ✅ **Docker Support** for easy deployment
-- ✅ **Extensive Testing** for reliability
-
-The system is now ready for production deployment and can serve as a foundation for further enhancements and integrations!

docs/GEOLOCATION_GUIDE.md

@@ -1,475 +0,0 @@
-# Geolocation Support for Grafana Geomap
-
-This guide explains the geolocation functionality added to the Thailand Water Monitor for use with Grafana's geomap visualization.
-
-## ✅ **Implemented Features**
-
-### **Database Schema Updates**
-All database adapters now support geolocation fields:
-- **latitude**: Decimal latitude coordinates (DECIMAL(10,8) for SQL, REAL for SQLite)
-- **longitude**: Decimal longitude coordinates (DECIMAL(11,8) for SQL, REAL for SQLite)  
-- **geohash**: Geohash string for efficient spatial indexing (VARCHAR(20)/TEXT)
-
-### **Station Data Enhancement**
-Station mapping now includes geolocation fields:
-```python
-'8': {
-    'code': 'P.1', 
-    'thai_name': 'สะพานนวรัฐ', 
-    'english_name': 'Nawarat Bridge',
-    'latitude': 15.6944,    # Decimal degrees
-    'longitude': 100.2028,  # Decimal degrees
-    'geohash': 'w5q6uuhvfcfp25'  # Geohash for P.1
-}
-```
-
-## 🗄️ **Database Schema**
-
-### **Updated Stations Table**
-```sql
-CREATE TABLE stations (
-    id INTEGER PRIMARY KEY,
-    station_code TEXT UNIQUE NOT NULL,
-    thai_name TEXT NOT NULL,
-    english_name TEXT NOT NULL,
-    latitude REAL,           -- NEW: Latitude coordinate
-    longitude REAL,          -- NEW: Longitude coordinate
-    geohash TEXT,           -- NEW: Geohash for spatial indexing
-    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-);
-```
-
-### **Database Support**
-- ✅ **SQLite**: REAL columns for coordinates, TEXT for geohash
-- ✅ **PostgreSQL**: DECIMAL(10,8) and DECIMAL(11,8) for coordinates, VARCHAR(20) for geohash
-- ✅ **MySQL**: DECIMAL(10,8) and DECIMAL(11,8) for coordinates, VARCHAR(20) for geohash
-- ✅ **VictoriaMetrics**: Geolocation data included in metric labels
-
-## 📊 **Current Station Data**
-
-### **P.1 - Nawarat Bridge (Sample)**
-- **Station Code**: P.1
-- **Thai Name**: สะพานนวรัฐ
-- **English Name**: Nawarat Bridge
-- **Latitude**: 15.6944
-- **Longitude**: 100.2028
-- **Geohash**: w5q6uuhvfcfp25
-
-### **Remaining Stations**
-The following stations are ready for geolocation data when coordinates become available:
-- P.20 - บ้านเชียงดาว (Ban Chiang Dao)
-- P.75 - บ้านช่อแล (Ban Chai Lat)
-- P.92 - บ้านเมืองกึ๊ด (Ban Muang Aut)
-- P.4A - บ้านแม่แตง (Ban Mae Taeng)
-- P.67 - บ้านแม่แต (Ban Tae)
-- P.21 - บ้านริมใต้ (Ban Rim Tai)
-- P.103 - สะพานวงแหวนรอบ 3 (Ring Bridge 3)
-- P.82 - บ้านสบวิน (Ban Sob win)
-- P.84 - บ้านพันตน (Ban Panton)
-- P.81 - บ้านโป่ง (Ban Pong)
-- P.5 - สะพานท่านาง (Tha Nang Bridge)
-- P.77 - บ้านสบแม่สะป๊วด (Baan Sop Mae Sapuord)
-- P.87 - บ้านป่าซาง (Ban Pa Sang)
-- P.76 - บ้านแม่อีไฮ (Banb Mae I Hai)
-- P.85 - บ้านหล่ายแก้ว (Baan Lai Kaew)
-
-## 🗺️ **Grafana Geomap Integration**
-
-### **Data Source Configuration**
-The geolocation data is automatically included in all database queries and can be used directly in Grafana:
-
-#### **SQLite/PostgreSQL/MySQL Query Example**
-```sql
-SELECT 
-    m.timestamp,
-    s.station_code,
-    s.english_name,
-    s.thai_name,
-    s.latitude,
-    s.longitude,
-    s.geohash,
-    m.water_level,
-    m.discharge,
-    m.discharge_percent
-FROM water_measurements m
-JOIN stations s ON m.station_id = s.id
-WHERE s.latitude IS NOT NULL 
-  AND s.longitude IS NOT NULL
-ORDER BY m.timestamp DESC
-```
-
-#### **VictoriaMetrics Query Example**
-```promql
-water_level{latitude!="",longitude!=""}
-```
-
-### **Geomap Panel Configuration**
-
-#### **1. Create Geomap Panel**
-1. Add new panel in Grafana
-2. Select "Geomap" visualization
-3. Configure data source (SQLite/PostgreSQL/MySQL/VictoriaMetrics)
-
-#### **2. Configure Location Fields**
-- **Latitude Field**: `latitude`
-- **Longitude Field**: `longitude`
-- **Alternative**: Use `geohash` field for geohash-based positioning
-
-#### **3. Configure Display Options**
-- **Station Labels**: Use `station_code` or `english_name`
-- **Tooltip Information**: Include `thai_name`, `water_level`, `discharge`
-- **Color Mapping**: Map to `water_level` or `discharge_percent`
-
-#### **4. Sample Geomap Configuration**
-```json
-{
-  "type": "geomap",
-  "title": "Thailand Water Stations",
-  "targets": [
-    {
-      "rawSql": "SELECT latitude, longitude, station_code, english_name, water_level, discharge_percent FROM stations s JOIN water_measurements m ON s.id = m.station_id WHERE s.latitude IS NOT NULL AND m.timestamp = (SELECT MAX(timestamp) FROM water_measurements WHERE station_id = s.id)",
-      "format": "table"
-    }
-  ],
-  "fieldConfig": {
-    "defaults": {
-      "custom": {
-        "hideFrom": {
-          "legend": false,
-          "tooltip": false,
-          "vis": false
-        }
-      },
-      "mappings": [],
-      "color": {
-        "mode": "continuous-GrYlRd",
-        "field": "water_level"
-      }
-    }
-  },
-  "options": {
-    "view": {
-      "id": "coords",
-      "lat": 15.6944,
-      "lon": 100.2028,
-      "zoom": 8
-    },
-    "controls": {
-      "mouseWheelZoom": true,
-      "showZoom": true,
-      "showAttribution": true
-    },
-    "layers": [
-      {
-        "type": "markers",
-        "config": {
-          "size": {
-            "field": "discharge_percent",
-            "min": 5,
-            "max": 20
-          },
-          "color": {
-            "field": "water_level"
-          },
-          "showLegend": true
-        }
-      }
-    ]
-  }
-}
-```
-
-## 🔧 **Adding New Station Coordinates**
-
-### **Method 1: Update Station Mapping**
-Edit `water_scraper_v3.py` and add coordinates to the station mapping:
-```python
-'1': {
-    'code': 'P.20', 
-    'thai_name': 'บ้านเชียงดาว', 
-    'english_name': 'Ban Chiang Dao',
-    'latitude': 19.3056,    # Add actual coordinates
-    'longitude': 98.9264,   # Add actual coordinates
-    'geohash': 'w4r6...'    # Add actual geohash
-}
-```
-
-### **Method 2: Direct Database Update**
-```sql
-UPDATE stations 
-SET latitude = 19.3056, longitude = 98.9264, geohash = 'w4r6uuhvfcfp25'
-WHERE station_code = 'P.20';
-```
-
-### **Method 3: Bulk Update Script**
-```python
-import sqlite3
-
-coordinates = {
-    'P.20': {'lat': 19.3056, 'lon': 98.9264, 'geohash': 'w4r6uuhvfcfp25'},
-    'P.75': {'lat': 18.7756, 'lon': 99.1234, 'geohash': 'w4r5uuhvfcfp25'},
-    # Add more stations...
-}
-
-conn = sqlite3.connect('water_monitoring.db')
-cursor = conn.cursor()
-
-for station_code, coords in coordinates.items():
-    cursor.execute("""
-        UPDATE stations 
-        SET latitude = ?, longitude = ?, geohash = ?
-        WHERE station_code = ?
-    """, (coords['lat'], coords['lon'], coords['geohash'], station_code))
-
-conn.commit()
-conn.close()
-```
-
-## 🌐 **Geohash Information**
-
-### **What is Geohash?**
-Geohash is a geocoding system that represents geographic coordinates as a short alphanumeric string. It provides:
-- **Spatial Indexing**: Efficient spatial queries
-- **Proximity**: Similar geohashes indicate nearby locations
-- **Hierarchical**: Longer geohashes provide more precision
-
-### **Geohash Precision Levels**
-- **5 characters**: ~2.4km precision
-- **6 characters**: ~610m precision  
-- **7 characters**: ~76m precision
-- **8 characters**: ~19m precision
-- **9+ characters**: <5m precision
-
-### **Example: P.1 Geohash**
-- **Geohash**: `w5q6uuhvfcfp25`
-- **Length**: 14 characters
-- **Precision**: Sub-meter accuracy
-- **Location**: Nawarat Bridge, Thailand
-
-## 📈 **Grafana Visualization Examples**
-
-### **1. Station Location Map**
-- **Type**: Geomap with markers
-- **Data**: Current station locations
-- **Color**: Water level or discharge percentage
-- **Size**: Discharge volume
-
-### **2. Regional Water Levels**
-- **Type**: Geomap with heatmap
-- **Data**: Water level data across regions
-- **Visualization**: Color-coded intensity map
-- **Filters**: Time range, station groups
-
-### **3. Alert Zones**
-- **Type**: Geomap with threshold markers
-- **Data**: Stations exceeding alert thresholds
-- **Visualization**: Red markers for high water levels
-- **Alerts**: Automated notifications for critical levels
-
-## 🔄 **Updating a Running System**
-
-### **Automated Migration Script**
-Use the provided migration script to safely add geolocation columns to your existing database:
-
-```bash
-# Stop the water monitoring service first
-sudo systemctl stop water-monitor
-
-# Run the migration script
-python migrate_geolocation.py
-
-# Restart the service
-sudo systemctl start water-monitor
-```
-
-### **Migration Script Features**
-- ✅ **Auto-detects database type** from environment variables
-- ✅ **Checks existing columns** to avoid conflicts
-- ✅ **Supports all database types** (SQLite, PostgreSQL, MySQL)
-- ✅ **Adds sample data** for P.1 station
-- ✅ **Safe operation** - won't break existing data
-
-### **Step-by-Step Migration Process**
-
-#### **1. Stop the Application**
-```bash
-# If running as systemd service
-sudo systemctl stop water-monitor
-
-# If running in screen/tmux
-# Use Ctrl+C to stop the process
-
-# If running as Docker container
-docker stop water-monitor
-```
-
-#### **2. Backup Your Database**
-```bash
-# SQLite backup
-cp water_monitoring.db water_monitoring.db.backup
-
-# PostgreSQL backup
-pg_dump water_monitoring > water_monitoring_backup.sql
-
-# MySQL backup
-mysqldump water_monitoring > water_monitoring_backup.sql
-```
-
-#### **3. Run Migration Script**
-```bash
-# Default (uses environment variables)
-python migrate_geolocation.py
-
-# Or specify database path for SQLite
-SQLITE_DB_PATH=/path/to/water_monitoring.db python migrate_geolocation.py
-```
-
-#### **4. Verify Migration**
-```bash
-# Check SQLite schema
-sqlite3 water_monitoring.db ".schema stations"
-
-# Check PostgreSQL schema
-psql -d water_monitoring -c "\d stations"
-
-# Check MySQL schema
-mysql -e "DESCRIBE water_monitoring.stations"
-```
-
-#### **5. Update Application Code**
-Ensure you have the latest version of the application with geolocation support:
-```bash
-# Pull latest code
-git pull origin main
-
-# Install any new dependencies
-pip install -r requirements.txt
-```
-
-#### **6. Restart Application**
-```bash
-# Systemd service
-sudo systemctl start water-monitor
-
-# Docker container
-docker start water-monitor
-
-# Manual execution
-python water_scraper_v3.py
-```
-
-### **Migration Output Example**
-```
-2025-07-28 17:30:00,123 - INFO - Starting geolocation column migration...
-2025-07-28 17:30:00,124 - INFO - Detected database type: SQLITE
-2025-07-28 17:30:00,125 - INFO - Migrating SQLite database: water_monitoring.db
-2025-07-28 17:30:00,126 - INFO - Current columns in stations table: ['id', 'station_code', 'thai_name', 'english_name', 'created_at', 'updated_at']
-2025-07-28 17:30:00,127 - INFO - Added latitude column
-2025-07-28 17:30:00,128 - INFO - Added longitude column
-2025-07-28 17:30:00,129 - INFO - Added geohash column
-2025-07-28 17:30:00,130 - INFO - Successfully added columns: latitude, longitude, geohash
-2025-07-28 17:30:00,131 - INFO - Updated P.1 station with sample geolocation data
-2025-07-28 17:30:00,132 - INFO - P.1 station geolocation: ('P.1', 15.6944, 100.2028, 'w5q6uuhvfcfp25')
-2025-07-28 17:30:00,133 - INFO - ✅ Migration completed successfully!
-2025-07-28 17:30:00,134 - INFO - You can now restart your water monitoring application
-2025-07-28 17:30:00,135 - INFO - The system will automatically use the new geolocation columns
-```
-
-## 🔍 **Troubleshooting**
-
-### **Migration Issues**
-
-#### **Database Locked Error**
-```bash
-# Stop all processes using the database
-sudo systemctl stop water-monitor
-pkill -f water_scraper
-
-# Wait a few seconds, then run migration
-sleep 5
-python migrate_geolocation.py
-```
-
-#### **Permission Denied**
-```bash
-# Check database file permissions
-ls -la water_monitoring.db
-
-# Fix permissions if needed
-sudo chown $USER:$USER water_monitoring.db
-chmod 664 water_monitoring.db
-```
-
-#### **Missing Dependencies**
-```bash
-# For PostgreSQL
-pip install psycopg2-binary
-
-# For MySQL
-pip install pymysql
-
-# For all databases
-pip install -r requirements.txt
-```
-
-### **Verification Issues**
-
-#### **Missing Coordinates**
-If stations don't appear on the geomap:
-1. Check if latitude/longitude are NULL in database
-2. Verify geolocation data in station mapping
-3. Ensure database schema includes geolocation columns
-4. Run migration script if columns are missing
-
-#### **Incorrect Positioning**
-If stations appear in wrong locations:
-1. Verify coordinate format (decimal degrees)
-2. Check latitude/longitude order (lat first, lon second)
-3. Validate geohash accuracy
-
-### **Rollback Procedure**
-If migration causes issues:
-
-#### **SQLite Rollback**
-```bash
-# Stop application
-sudo systemctl stop water-monitor
-
-# Restore backup
-cp water_monitoring.db.backup water_monitoring.db
-
-# Restart with old version
-sudo systemctl start water-monitor
-```
-
-#### **PostgreSQL Rollback**
-```sql
--- Remove added columns
-ALTER TABLE stations DROP COLUMN IF EXISTS latitude;
-ALTER TABLE stations DROP COLUMN IF EXISTS longitude;
-ALTER TABLE stations DROP COLUMN IF EXISTS geohash;
-```
-
-#### **MySQL Rollback**
-```sql
--- Remove added columns
-ALTER TABLE stations DROP COLUMN latitude;
-ALTER TABLE stations DROP COLUMN longitude;
-ALTER TABLE stations DROP COLUMN geohash;
-```
-
-## 🎯 **Next Steps**
-
-### **Immediate Actions**
-1. **Gather Coordinates**: Collect GPS coordinates for all 16 stations
-2. **Update Database**: Add coordinates to remaining stations
-3. **Create Dashboards**: Build Grafana geomap visualizations
-
-### **Future Enhancements**
-1. **Automatic Geocoding**: API integration for address-to-coordinate conversion
-2. **Mobile GPS**: Mobile app for field coordinate collection
-3. **Satellite Integration**: Satellite imagery overlay in Grafana
-4. **Geofencing**: Alert zones based on geographic boundaries
-
-The geolocation functionality is now fully implemented and ready for use with Grafana's geomap visualization. Station P.1 (Nawarat Bridge) serves as a working example with complete coordinate data.

docs/GITEA_WORKFLOWS.md

@@ -286,8 +286,6 @@ make validate-workflows
 
 ### **Project-Specific Resources**
 - [Contributing Guide](../CONTRIBUTING.md)
-- [Deployment Checklist](../DEPLOYMENT_CHECKLIST.md)
-- [Project Structure](PROJECT_STRUCTURE.md)
 
 ### **Monitoring and Alerts**
 - Workflow status badges in README

docs/GRAFANA_MATRIX_ALERTING.md

@@ -1,168 +0,0 @@
-# Grafana Matrix Alerting Setup
-
-## Overview
-Configure Grafana to send water level alerts directly to Matrix channels when thresholds are exceeded.
-
-## Prerequisites
-- Grafana instance with your PostgreSQL data source
-- Matrix account and access token
-- Matrix room for alerts
-
-## Step 1: Configure Matrix Contact Point
-
-1. **In Grafana, go to Alerting → Contact Points**
-2. **Add new contact point:**
-   ```
-   Name: matrix-water-alerts
-   Integration: Webhook
-   URL: https://matrix.org/_matrix/client/v3/rooms/!ROOM_ID:matrix.org/send/m.room.message
-   HTTP Method: POST
-   ```
-
-3. **Add Headers:**
-   ```
-   Authorization: Bearer YOUR_MATRIX_ACCESS_TOKEN
-   Content-Type: application/json
-   ```
-
-4. **Message Template:**
-   ```json
-   {
-     "msgtype": "m.text",
-     "body": "🌊 WATER ALERT: {{ .CommonLabels.alertname }}\n\nStation: {{ .CommonLabels.station_code }}\nLevel: {{ .CommonAnnotations.water_level }}m\nStatus: {{ .CommonLabels.severity }}\n\nTime: {{ .CommonAnnotations.time }}"
-   }
-   ```
-
-## Step 2: Create Alert Rules
-
-### High Water Level Alert
-```yaml
-Rule Name: high-water-level
-Query: water_level > 6.0
-Condition: IS ABOVE 6.0 FOR 5m
-Labels:
-  - severity: critical
-  - station_code: {{ .station_code }}
-Annotations:
-  - water_level: {{ .water_level }}
-  - summary: "Critical water level at {{ .station_code }}"
-```
-
-### Low Water Level Alert
-```yaml
-Rule Name: low-water-level
-Query: water_level < 1.0
-Condition: IS BELOW 1.0 FOR 10m
-Labels:
-  - severity: warning
-  - station_code: {{ .station_code }}
-```
-
-### Data Gap Alert
-```yaml
-Rule Name: data-gap
-Query: increase(measurements_total[1h]) == 0
-Condition: IS EQUAL TO 0 FOR 30m
-Labels:
-  - severity: warning
-  - issue: data-gap
-```
-
-## Step 3: Matrix Setup
-
-### Get Matrix Access Token
-```bash
-curl -X POST https://matrix.org/_matrix/client/v3/login \
-  -H "Content-Type: application/json" \
-  -d '{
-    "type": "m.login.password",
-    "user": "your_username",
-    "password": "your_password"
-  }'
-```
-
-### Create Alert Room
-```bash
-curl -X POST "https://matrix.org/_matrix/client/v3/createRoom" \
-  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "name": "Water Level Alerts - Northern Thailand",
-    "topic": "Automated alerts for Ping River water monitoring",
-    "preset": "trusted_private_chat"
-  }'
-```
-
-## Example Alert Queries
-
-### Critical Water Levels
-```promql
-# High water alert
-water_level{station_code=~"P.1|P.4A|P.20"} > 6.0
-
-# Dangerous discharge
-discharge{station_code=~".*"} > 500
-
-# Rapid level change
-increase(water_level[15m]) > 0.5
-```
-
-### System Health
-```promql
-# No data received
-up{job="water-monitor"} == 0
-
-# Old data
-(time() - timestamp) > 7200
-```
-
-## Alert Notification Format
-
-Your Matrix messages will look like:
-```
-🌊 WATER ALERT: High Water Level
-
-Station: P.1 (Chiang Mai)
-Level: 6.2m (CRITICAL)
-Discharge: 450 cms
-Status: DANGER
-
-Time: 2025-09-26 14:30:00
-Trend: Rising (+0.3m in 30min)
-
-📍 Location: 18.7883°N, 98.9853°E
-```
-
-## Advanced Features
-
-### Escalation Rules
-```yaml
-# Send to different rooms based on severity
-- if: severity == "critical"
-  receiver: matrix-emergency
-- if: severity == "warning"
-  receiver: matrix-alerts
-- if: time_of_day() outside "08:00-20:00"
-  receiver: matrix-night-duty
-```
-
-### Rate Limiting
-```yaml
-group_wait: 5m
-group_interval: 10m
-repeat_interval: 30m
-```
-
-## Testing Alerts
-
-1. **Test Contact Point** - Use Grafana's test button
-2. **Simulate Alert** - Manually trigger with test data
-3. **Verify Matrix** - Check message formatting and delivery
-
-## Troubleshooting
-
-### Common Issues
-- **403 Forbidden**: Check Matrix access token
-- **Room not found**: Verify room ID format
-- **No alerts**: Check query syntax and thresholds
-- **Spam**: Configure proper grouping and intervals

docs/GRAFANA_MATRIX_SETUP.md

@@ -1,351 +0,0 @@
-# Complete Grafana Matrix Alerting Setup Guide
-
-## Overview
-Configure Grafana to send water level alerts directly to Matrix channels when thresholds are exceeded.
-
-## Prerequisites
-- Grafana instance running (v8.0+)
-- PostgreSQL data source configured in Grafana
-- Matrix account
-- Matrix room for alerts
-
-## Step 1: Get Matrix Access Token
-
-### Method 1: Using curl
-```bash
-curl -X POST https://matrix.org/_matrix/client/v3/login \
-  -H "Content-Type: application/json" \
-  -d '{
-    "type": "m.login.password",
-    "user": "your_username",
-    "password": "your_password"
-  }'
-```
-
-### Method 2: Using Element Web Client
-1. Open Element in browser: https://app.element.io
-2. Login to your account
-3. Go to Settings → Help & About → Advanced
-4. Copy your Access Token
-
-### Method 3: Using Matrix Admin Panel
-- If you have admin access to your homeserver, generate token via admin API
-
-## Step 2: Create Alert Room
-
-```bash
-curl -X POST "https://matrix.org/_matrix/client/v3/createRoom" \
-  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "name": "Water Level Alerts - Northern Thailand",
-    "topic": "Automated alerts for Ping River water monitoring",
-    "preset": "private_chat"
-  }'
-```
-
-Save the `room_id` from the response (format: !roomid:homeserver.com)
-
-## Step 3: Configure Grafana Contact Point
-
-### Navigate to Alerting
-1. In Grafana, go to **Alerting → Contact Points**
-2. Click **Add contact point**
-
-### Contact Point Settings
-```
-Name: matrix-water-alerts
-Integration: Webhook
-URL: https://matrix.org/_matrix/client/v3/rooms/!YOUR_ROOM_ID:matrix.org/send/m.room.message/{{ .GroupLabels.alertname }}_{{ .GroupLabels.severity }}_{{ now.Unix }}
-HTTP Method: POST
-```
-
-### Headers
-```
-Authorization: Bearer YOUR_MATRIX_ACCESS_TOKEN
-Content-Type: application/json
-```
-
-### Message Template (JSON Body)
-```json
-{
-  "msgtype": "m.text",
-  "body": "🌊 **PING RIVER WATER ALERT**\n\n**Alert:** {{ .GroupLabels.alertname }}\n**Severity:** {{ .GroupLabels.severity | toUpper }}\n**Station:** {{ .GroupLabels.station_code }} ({{ .GroupLabels.station_name }})\n\n{{ range .Alerts }}**Status:** {{ .Status | toUpper }}\n**Water Level:** {{ .Annotations.water_level }}m\n**Threshold:** {{ .Annotations.threshold }}m\n**Time:** {{ .StartsAt.Format \"2006-01-02 15:04:05\" }}\n{{ if .Annotations.discharge }}**Discharge:** {{ .Annotations.discharge }} cms\n{{ end }}{{ if .Annotations.message }}**Details:** {{ .Annotations.message }}\n{{ end }}{{ end }}\n📈 **Dashboard:** {{ .ExternalURL }}\n📍 **Location:** Northern Thailand Ping River"
-}
-```
-
-## Step 4: Create Alert Rules
-
-### High Water Level Alert
-```yaml
-# Rule Configuration
-Rule Name: high-water-level
-Evaluation Group: water-level-alerts
-Folder: Water Monitoring
-
-# Query A
-SELECT
-  station_code,
-  station_name_th as station_name,
-  water_level,
-  discharge,
-  timestamp
-FROM water_measurements
-WHERE
-  timestamp > now() - interval '5 minutes'
-  AND water_level > 6.0
-
-# Condition
-IS ABOVE 6.0 FOR 5 minutes
-
-# Labels
-severity: critical
-alertname: High Water Level
-station_code: {{ $labels.station_code }}
-station_name: {{ $labels.station_name }}
-
-# Annotations
-water_level: {{ $values.water_level }}
-threshold: 6.0
-discharge: {{ $values.discharge }}
-summary: Critical water level detected at {{ $labels.station_code }}
-```
-
-### Emergency Water Level Alert
-```yaml
-Rule Name: emergency-water-level
-Query: water_level > 8.0
-Condition: IS ABOVE 8.0 FOR 2 minutes
-Labels:
-  severity: emergency
-  alertname: Emergency Water Level
-Annotations:
-  threshold: 8.0
-  message: IMMEDIATE ACTION REQUIRED - Flood risk imminent
-```
-
-### Low Water Level Alert
-```yaml
-Rule Name: low-water-level
-Query: water_level < 1.0
-Condition: IS BELOW 1.0 FOR 15 minutes
-Labels:
-  severity: warning
-  alertname: Low Water Level
-Annotations:
-  threshold: 1.0
-  message: Drought conditions detected
-```
-
-### Data Gap Alert
-```yaml
-Rule Name: data-gap
-Query:
-  SELECT
-    station_code,
-    MAX(timestamp) as last_seen
-  FROM water_measurements
-  GROUP BY station_code
-  HAVING MAX(timestamp) < now() - interval '2 hours'
-
-Condition: HAS NO DATA FOR 30 minutes
-Labels:
-  severity: warning
-  alertname: Data Gap
-  issue: missing-data
-```
-
-### Rapid Level Change Alert
-```yaml
-Rule Name: rapid-level-change
-Query:
-  SELECT
-    station_code,
-    water_level,
-    LAG(water_level, 1) OVER (PARTITION BY station_code ORDER BY timestamp) as prev_level
-  FROM water_measurements
-  WHERE timestamp > now() - interval '15 minutes'
-  HAVING ABS(water_level - prev_level) > 0.5
-
-Condition: CHANGE > 0.5m FOR 1 minute
-Labels:
-  severity: warning
-  alertname: Rapid Water Level Change
-```
-
-## Step 5: Configure Notification Policy
-
-### Create Notification Policy
-```yaml
-# Policy Tree
-- receiver: matrix-water-alerts
-  match:
-    severity: emergency|critical
-  group_wait: 10s
-  group_interval: 5m
-  repeat_interval: 30m
-
-- receiver: matrix-water-alerts
-  match:
-    severity: warning
-  group_wait: 30s
-  group_interval: 10m
-  repeat_interval: 2h
-```
-
-### Grouping Rules
-```yaml
-group_by: [alertname, station_code]
-group_wait: 10s
-group_interval: 5m
-repeat_interval: 1h
-```
-
-## Step 6: Station-Specific Thresholds
-
-Create separate rules for each station with appropriate thresholds:
-
-```sql
--- P.1 (Chiang Mai) - Urban area, higher thresholds
-SELECT * FROM water_measurements
-WHERE station_code = 'P.1' AND water_level > 6.5
-
--- P.4A (Mae Ping) - Agricultural area
-SELECT * FROM water_measurements
-WHERE station_code = 'P.4A' AND water_level > 5.0
-
--- P.20 (Downstream) - Lower threshold
-SELECT * FROM water_measurements
-WHERE station_code = 'P.20' AND water_level > 4.0
-```
-
-## Step 7: Advanced Features
-
-### Time-Based Routing
-```yaml
-# Different receivers for day/night
-time_intervals:
-  - name: working_hours
-    time_intervals:
-      - times:
-        - start_time: '08:00'
-          end_time: '20:00'
-        weekdays: ['monday:friday']
-
-routes:
-  - receiver: matrix-alerts-day
-    match:
-      severity: warning
-    active_time_intervals: [working_hours]
-
-  - receiver: matrix-alerts-night
-    match:
-      severity: warning
-    active_time_intervals: ['!working_hours']
-```
-
-### Multi-Channel Alerts
-```yaml
-# Send critical alerts to multiple rooms
-- receiver: matrix-emergency
-  webhook_configs:
-    - url: https://matrix.org/_matrix/client/v3/rooms/!emergency:matrix.org/send/m.room.message
-      http_config:
-        authorization:
-          credentials: "Bearer EMERGENCY_TOKEN"
-    - url: https://matrix.org/_matrix/client/v3/rooms/!general:matrix.org/send/m.room.message
-      http_config:
-        authorization:
-          credentials: "Bearer GENERAL_TOKEN"
-```
-
-## Step 8: Testing
-
-### Test Contact Point
-1. Go to Contact Points in Grafana
-2. Select your Matrix contact point
-3. Click "Test" button
-4. Check Matrix room for test message
-
-### Test Alert Rules
-1. Temporarily lower thresholds
-2. Wait for condition to trigger
-3. Verify alert appears in Grafana
-4. Verify Matrix message received
-5. Reset thresholds
-
-### Manual Alert Trigger
-```bash
-# Simulate high water level in database
-INSERT INTO water_measurements (station_code, water_level, timestamp)
-VALUES ('P.1', 7.5, NOW());
-```
-
-## Troubleshooting
-
-### Common Issues
-
-#### 403 Forbidden
-- **Cause**: Invalid Matrix access token
-- **Fix**: Regenerate token or check permissions
-
-#### Room Not Found
-- **Cause**: Incorrect room ID format
-- **Fix**: Ensure room ID starts with ! and includes homeserver
-
-#### No Alerts Firing
-- **Cause**: Query returns no results
-- **Fix**: Test queries in Grafana Explore, check data availability
-
-#### Alert Spam
-- **Cause**: No grouping configured
-- **Fix**: Configure proper group_by and intervals
-
-#### Messages Not Formatted
-- **Cause**: Template syntax errors
-- **Fix**: Validate JSON template, check Grafana template docs
-
-### Debug Steps
-1. Check Grafana alert rule status
-2. Verify contact point test succeeds
-3. Check Grafana logs: `/var/log/grafana/grafana.log`
-4. Test Matrix API directly with curl
-5. Verify database connectivity and query results
-
-## Environment Variables
-
-Add to your `.env`:
-```bash
-MATRIX_HOMESERVER=https://matrix.org
-MATRIX_ACCESS_TOKEN=your_access_token_here
-MATRIX_ROOM_ID=!your_room_id:matrix.org
-GRAFANA_URL=http://your-grafana-host:3000
-```
-
-## Example Alert Message
-Your Matrix messages will appear as:
-```
-🌊 **PING RIVER WATER ALERT**
-
-**Alert:** High Water Level
-**Severity:** CRITICAL
-**Station:** P.1 (สถานีเชียงใหม่)
-
-**Status:** FIRING
-**Water Level:** 6.75m
-**Threshold:** 6.0m
-**Time:** 2025-09-26 14:30:00
-**Discharge:** 450.2 cms
-
-📈 **Dashboard:** http://grafana:3000
-📍 **Location:** Northern Thailand Ping River
-```
-
-## Security Notes
-- Store Matrix tokens securely (environment variables)
-- Use room-specific tokens when possible
-- Enable rate limiting to prevent spam
-- Consider using dedicated alerting user account
-- Regularly rotate access tokens
-
-This setup provides comprehensive water level monitoring with immediate Matrix notifications when thresholds are exceeded.

docs/HTTPS_CONFIGURATION.md

@@ -1,389 +0,0 @@
-# HTTPS VictoriaMetrics Configuration Guide
-
-This guide explains how to configure the Thailand Water Monitor to connect to VictoriaMetrics through HTTPS and reverse proxies.
-
-## Configuration Options
-
-### 1. Environment Variables for HTTPS
-
-```bash
-# Option 1: Full HTTPS URL (Recommended)
-export DB_TYPE=victoriametrics
-export VM_HOST=https://vm.example.com
-export VM_PORT=443
-
-# Option 2: Host and port separately
-export DB_TYPE=victoriametrics
-export VM_HOST=vm.example.com
-export VM_PORT=443
-
-# Option 3: Custom port with HTTPS
-export DB_TYPE=victoriametrics
-export VM_HOST=https://vm.example.com
-export VM_PORT=8443
-```
-
-### 2. Windows PowerShell Configuration
-
-```powershell
-# Set environment variables for HTTPS
-$env:DB_TYPE="victoriametrics"
-$env:VM_HOST="https://vm.example.com"
-$env:VM_PORT="443"
-
-# Run the water monitor
-python water_scraper_v3.py
-```
-
-### 3. Linux/Mac Configuration
-
-```bash
-# Set environment variables for HTTPS
-export DB_TYPE=victoriametrics
-export VM_HOST=https://vm.example.com
-export VM_PORT=443
-
-# Run the water monitor
-python water_scraper_v3.py
-```
-
-## Reverse Proxy Examples
-
-### 1. Nginx Reverse Proxy
-
-```nginx
-server {
-    listen 443 ssl http2;
-    server_name vm.example.com;
-    
-    # SSL Configuration
-    ssl_certificate /path/to/certificate.crt;
-    ssl_certificate_key /path/to/private.key;
-    ssl_protocols TLSv1.2 TLSv1.3;
-    ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512;
-    
-    # Security headers
-    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
-    add_header X-Frame-Options DENY always;
-    add_header X-Content-Type-Options nosniff always;
-    
-    # Optional: Basic authentication
-    # auth_basic "VictoriaMetrics";
-    # auth_basic_user_file /etc/nginx/.htpasswd;
-    
-    location / {
-        proxy_pass http://localhost:8428;
-        proxy_set_header Host $host;
-        proxy_set_header X-Real-IP $remote_addr;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header X-Forwarded-Proto $scheme;
-        
-        # WebSocket support (if needed)
-        proxy_http_version 1.1;
-        proxy_set_header Upgrade $http_upgrade;
-        proxy_set_header Connection "upgrade";
-        
-        # Timeouts
-        proxy_connect_timeout 60s;
-        proxy_send_timeout 60s;
-        proxy_read_timeout 60s;
-    }
-}
-
-# Redirect HTTP to HTTPS
-server {
-    listen 80;
-    server_name vm.example.com;
-    return 301 https://$server_name$request_uri;
-}
-```
-
-### 2. Apache Reverse Proxy
-
-```apache
-<VirtualHost *:443>
-    ServerName vm.example.com
-    
-    # SSL Configuration
-    SSLEngine on
-    SSLCertificateFile /path/to/certificate.crt
-    SSLCertificateKeyFile /path/to/private.key
-    SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
-    SSLCipherSuite ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384
-    
-    # Security headers
-    Header always set Strict-Transport-Security "max-age=31536000; includeSubDomains"
-    Header always set X-Frame-Options DENY
-    Header always set X-Content-Type-Options nosniff
-    
-    # Reverse proxy configuration
-    ProxyPreserveHost On
-    ProxyPass / http://localhost:8428/
-    ProxyPassReverse / http://localhost:8428/
-    
-    # Optional: Basic authentication
-    # AuthType Basic
-    # AuthName "VictoriaMetrics"
-    # AuthUserFile /etc/apache2/.htpasswd
-    # Require valid-user
-</VirtualHost>
-
-<VirtualHost *:80>
-    ServerName vm.example.com
-    Redirect permanent / https://vm.example.com/
-</VirtualHost>
-```
-
-### 3. Traefik Reverse Proxy
-
-```yaml
-# docker-compose.yml with Traefik
-version: '3.8'
-
-services:
-  traefik:
-    image: traefik:v2.10
-    command:
-      - --api.dashboard=true
-      - --entrypoints.web.address=:80
-      - --entrypoints.websecure.address=:443
-      - --providers.docker=true
-      - --certificatesresolvers.letsencrypt.acme.tlschallenge=true
-      - --certificatesresolvers.letsencrypt.acme.email=admin@example.com
-      - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
-    ports:
-      - "80:80"
-      - "443:443"
-    volumes:
-      - /var/run/docker.sock:/var/run/docker.sock
-      - letsencrypt:/letsencrypt
-    labels:
-      - traefik.http.routers.api.rule=Host(`traefik.example.com`)
-      - traefik.http.routers.api.tls.certresolver=letsencrypt
-
-  victoriametrics:
-    image: victoriametrics/victoria-metrics:latest
-    command:
-      - '--storageDataPath=/victoria-metrics-data'
-      - '--retentionPeriod=2y'
-      - '--httpListenAddr=:8428'
-    volumes:
-      - vm_data:/victoria-metrics-data
-    labels:
-      - traefik.enable=true
-      - traefik.http.routers.vm.rule=Host(`vm.example.com`)
-      - traefik.http.routers.vm.tls.certresolver=letsencrypt
-      - traefik.http.services.vm.loadbalancer.server.port=8428
-
-volumes:
-  vm_data:
-  letsencrypt:
-```
-
-## Testing HTTPS Configuration
-
-### 1. Test Connection
-
-```bash
-# Test HTTPS connection
-curl -k https://vm.example.com/health
-
-# Test with specific port
-curl -k https://vm.example.com:8443/health
-
-# Test API endpoint
-curl -k "https://vm.example.com/api/v1/query?query=up"
-```
-
-### 2. Test with Water Monitor
-
-```bash
-# Set environment variables
-export DB_TYPE=victoriametrics
-export VM_HOST=https://vm.example.com
-export VM_PORT=443
-
-# Test with demo script
-python demo_databases.py victoriametrics
-
-# Run full water monitor
-python water_scraper_v3.py
-```
-
-### 3. Verify SSL Certificate
-
-```bash
-# Check SSL certificate
-openssl s_client -connect vm.example.com:443 -servername vm.example.com
-
-# Check certificate expiration
-echo | openssl s_client -connect vm.example.com:443 2>/dev/null | openssl x509 -noout -dates
-```
-
-## Configuration Examples
-
-### 1. Production HTTPS Setup
-
-```bash
-# Environment variables for production
-export DB_TYPE=victoriametrics
-export VM_HOST=https://metrics.company.com
-export VM_PORT=443
-export LOG_LEVEL=INFO
-export SCRAPING_INTERVAL_HOURS=1
-
-# Run water monitor
-python water_scraper_v3.py
-```
-
-### 2. Development with Self-Signed Certificate
-
-```bash
-# For development with self-signed certificates
-export DB_TYPE=victoriametrics
-export VM_HOST=https://dev-vm.local
-export VM_PORT=443
-export PYTHONHTTPSVERIFY=0  # Disable SSL verification (dev only)
-
-python water_scraper_v3.py
-```
-
-### 3. Custom Port Configuration
-
-```bash
-# Custom HTTPS port
-export DB_TYPE=victoriametrics
-export VM_HOST=https://vm.example.com
-export VM_PORT=8443
-
-python water_scraper_v3.py
-```
-
-## Troubleshooting HTTPS Issues
-
-### 1. SSL Certificate Errors
-
-```bash
-# Error: SSL certificate verify failed
-# Solution: Check certificate validity
-openssl x509 -in certificate.crt -text -noout
-
-# Temporary workaround (not recommended for production)
-export PYTHONHTTPSVERIFY=0
-```
-
-### 2. Connection Timeout
-
-```bash
-# Error: Connection timeout
-# Check firewall and network connectivity
-telnet vm.example.com 443
-nc -zv vm.example.com 443
-```
-
-### 3. DNS Resolution Issues
-
-```bash
-# Error: Name resolution failed
-# Check DNS resolution
-nslookup vm.example.com
-dig vm.example.com
-```
-
-### 4. Proxy Configuration Issues
-
-```bash
-# Check proxy logs
-# Nginx
-tail -f /var/log/nginx/error.log
-
-# Apache
-tail -f /var/log/apache2/error.log
-
-# Test direct connection to backend
-curl http://localhost:8428/health
-```
-
-## Security Best Practices
-
-### 1. SSL/TLS Configuration
-
-- Use TLS 1.2 or higher
-- Disable weak ciphers
-- Enable HSTS headers
-- Use strong SSL certificates
-
-### 2. Authentication
-
-```nginx
-# Basic authentication in Nginx
-auth_basic "VictoriaMetrics Access";
-auth_basic_user_file /etc/nginx/.htpasswd;
-
-# Create password file
-htpasswd -c /etc/nginx/.htpasswd username
-```
-
-### 3. Network Security
-
-- Use firewall rules to restrict access
-- Consider VPN for internal access
-- Implement rate limiting
-- Monitor access logs
-
-### 4. Certificate Management
-
-```bash
-# Auto-renewal with Let's Encrypt
-certbot renew --dry-run
-
-# Certificate monitoring
-echo | openssl s_client -connect vm.example.com:443 2>/dev/null | \
-openssl x509 -noout -dates | grep notAfter
-```
-
-## Docker Configuration for HTTPS
-
-### 1. Docker Compose with HTTPS
-
-```yaml
-version: '3.8'
-
-services:
-  water-monitor:
-    build: .
-    environment:
-      - DB_TYPE=victoriametrics
-      - VM_HOST=https://vm.example.com
-      - VM_PORT=443
-    restart: unless-stopped
-    depends_on:
-      - victoriametrics
-
-  victoriametrics:
-    image: victoriametrics/victoria-metrics:latest
-    ports:
-      - "8428:8428"
-    volumes:
-      - vm_data:/victoria-metrics-data
-    command:
-      - '--storageDataPath=/victoria-metrics-data'
-      - '--retentionPeriod=2y'
-      - '--httpListenAddr=:8428'
-
-volumes:
-  vm_data:
-```
-
-### 2. Environment File (.env)
-
-```bash
-# .env file
-DB_TYPE=victoriametrics
-VM_HOST=https://vm.example.com
-VM_PORT=443
-LOG_LEVEL=INFO
-SCRAPING_INTERVAL_HOURS=1
-```
-
-This configuration guide provides comprehensive instructions for setting up HTTPS connectivity to VictoriaMetrics through reverse proxies, ensuring secure and reliable data transmission for the Thailand Water Monitor.

docs/MIGRATION_QUICKSTART.md

@@ -1,136 +0,0 @@
-# Geolocation Migration Quick Start
-
-This is a quick reference guide for updating a running Thailand Water Monitor system to add geolocation support for Grafana geomap.
-
-## 🚀 **Quick Migration (5 minutes)**
-
-### **Step 1: Stop Application**
-```bash
-# Stop the service (choose your method)
-sudo systemctl stop water-monitor
-# OR
-docker stop water-monitor
-# OR use Ctrl+C if running manually
-```
-
-### **Step 2: Backup Database**
-```bash
-# SQLite backup
-cp water_monitoring.db water_monitoring.db.backup
-
-# PostgreSQL backup
-pg_dump water_monitoring > backup.sql
-
-# MySQL backup
-mysqldump water_monitoring > backup.sql
-```
-
-### **Step 3: Run Migration**
-```bash
-# Run the automated migration script
-python migrate_geolocation.py
-```
-
-### **Step 4: Restart Application**
-```bash
-# Restart the service
-sudo systemctl start water-monitor
-# OR
-docker start water-monitor
-# OR
-python water_scraper_v3.py
-```
-
-## ✅ **Expected Output**
-```
-2025-07-28 17:30:00,123 - INFO - Starting geolocation column migration...
-2025-07-28 17:30:00,124 - INFO - Detected database type: SQLITE
-2025-07-28 17:30:00,127 - INFO - Added latitude column
-2025-07-28 17:30:00,128 - INFO - Added longitude column
-2025-07-28 17:30:00,129 - INFO - Added geohash column
-2025-07-28 17:30:00,133 - INFO - ✅ Migration completed successfully!
-```
-
-## 🗺️ **Verify Geolocation Works**
-
-### **Check Database**
-```bash
-# SQLite
-sqlite3 water_monitoring.db "SELECT station_code, latitude, longitude, geohash FROM stations WHERE station_code = 'P.1';"
-
-# Expected output: P.1|15.6944|100.2028|w5q6uuhvfcfp25
-```
-
-### **Test Application**
-```bash
-# Run a test cycle
-python water_scraper_v3.py --test
-
-# Should complete without errors
-```
-
-## 🔧 **Grafana Setup**
-
-### **Query for Geomap**
-```sql
-SELECT 
-    s.latitude, s.longitude, s.station_code, s.english_name,
-    m.water_level, m.discharge_percent
-FROM stations s 
-JOIN water_measurements m ON s.id = m.station_id
-WHERE s.latitude IS NOT NULL 
-  AND m.timestamp = (SELECT MAX(timestamp) FROM water_measurements WHERE station_id = s.id)
-```
-
-### **Geomap Configuration**
-1. Create new panel → Select "Geomap"
-2. Set **Latitude field**: `latitude`
-3. Set **Longitude field**: `longitude`
-4. Set **Color field**: `water_level`
-5. Set **Size field**: `discharge_percent`
-
-## 🚨 **Troubleshooting**
-
-### **Database Locked**
-```bash
-sudo systemctl stop water-monitor
-pkill -f water_scraper
-sleep 5
-python migrate_geolocation.py
-```
-
-### **Permission Error**
-```bash
-sudo chown $USER:$USER water_monitoring.db
-chmod 664 water_monitoring.db
-```
-
-### **Missing Dependencies**
-```bash
-pip install psycopg2-binary pymysql
-```
-
-## 🔄 **Rollback (if needed)**
-```bash
-# Stop application
-sudo systemctl stop water-monitor
-
-# Restore backup
-cp water_monitoring.db.backup water_monitoring.db
-
-# Restart
-sudo systemctl start water-monitor
-```
-
-## 📚 **More Information**
-- **Full Guide**: See `GEOLOCATION_GUIDE.md`
-- **Migration Script**: `migrate_geolocation.py`
-- **Database Schema**: Updated with latitude, longitude, geohash columns
-
-## 🎯 **What You Get**
-- ✅ **P.1 Station** ready for geomap (Nawarat Bridge)
-- ✅ **Database Schema** updated for all 16 stations
-- ✅ **Grafana Compatible** data structure
-- ✅ **Backward Compatible** - existing data preserved
-
-**Total Time**: ~5 minutes for complete migration

docs/PROJECT_STATUS.md

@@ -1,206 +0,0 @@
-# Thailand Water Monitor - Current Project Status
-
-## 📁 **Clean Project Structure**
-
-The project has been cleaned up and organized with the following structure:
-
-```
-water_level_monitor/
-├── 📄 .gitignore                             # Git ignore rules
-├── 📄 README.md                              # Main project documentation
-├── 📄 requirements.txt                       # Python dependencies
-├── 📄 config.py                              # Configuration management
-├── 📄 water_scraper_v3.py                    # Main application (15-min scheduler)
-├── 📄 database_adapters.py                   # Multi-database support
-├── 📄 demo_databases.py                      # Database demonstration
-├── 📄 Dockerfile                             # Container configuration
-├── 📄 docker-compose.victoriametrics.yml     # VictoriaMetrics stack
-├── 📚 Documentation/
-│   ├── 📄 DATABASE_DEPLOYMENT_GUIDE.md       # Multi-database setup guide
-│   ├── 📄 DEBIAN_TROUBLESHOOTING.md          # Linux deployment guide
-│   ├── 📄 ENHANCED_SCHEDULER_GUIDE.md        # 15-minute scheduler guide
-│   ├── 📄 GAP_FILLING_GUIDE.md              # Data gap filling guide
-│   ├── 📄 HTTPS_CONFIGURATION.md            # HTTPS setup guide
-│   └── 📄 VICTORIAMETRICS_SETUP.md          # VictoriaMetrics guide
-└── 📁 grafana/                              # Grafana configuration
-    ├── 📁 provisioning/
-    │   ├── 📁 datasources/
-    │   │   └── 📄 victoriametrics.yml        # VictoriaMetrics data source
-    │   └── 📁 dashboards/
-    │       └── 📄 dashboard.yml              # Dashboard provider config
-    └── 📁 dashboards/
-        └── 📄 water-monitoring-dashboard.json # Pre-built dashboard
-```
-
-## 🧹 **Files Removed During Cleanup**
-
-### **Old Data Files**
-- ❌ `thailand_water_data_v2.csv` - Old CSV export
-- ❌ `water_monitor.log` - Log file (regenerated automatically)
-- ❌ `water_monitoring.db` - SQLite database (recreated automatically)
-
-### **Outdated Documentation**
-- ❌ `FINAL_SUMMARY.md` - Contained references to non-existent v2 files
-- ❌ `PROJECT_SUMMARY.md` - Outdated project information
-
-### **System Files**
-- ❌ `__pycache__/` - Python compiled files directory
-
-## ✅ **Current Features**
-
-### **Enhanced 15-Minute Scheduler**
-- **Timing**: Runs every 15 minutes (1:00, 1:15, 1:30, 1:45, 2:00, etc.)
-- **Full Checks**: At :00 minutes (gap filling + data updates)
-- **Quick Checks**: At :15, :30, :45 minutes (data fetch only)
-- **Gap Filling**: Automatically fills missing historical data
-- **Data Updates**: Updates existing records when values change
-
-### **Multi-Database Support**
-- **VictoriaMetrics** (Recommended) - High-performance time-series
-- **InfluxDB** - Purpose-built time-series database
-- **PostgreSQL + TimescaleDB** - Relational with time-series optimization
-- **MySQL** - Traditional relational database
-- **SQLite** - Local development and testing
-
-### **Production Features**
-- **Docker Support**: Complete containerization
-- **Grafana Integration**: Pre-built dashboards
-- **HTTPS Configuration**: Secure deployment options
-- **Health Monitoring**: Comprehensive logging and error handling
-- **Gap Detection**: Automatic identification of missing data
-- **Retry Logic**: Database lock handling and network error recovery
-
-## 🚀 **Quick Start**
-
-### **1. Basic Setup (SQLite)**
-```bash
-cd water_level_monitor
-pip install -r requirements.txt
-python water_scraper_v3.py
-```
-
-### **2. VictoriaMetrics Setup**
-```bash
-# Start VictoriaMetrics + Grafana
-docker-compose -f docker-compose.victoriametrics.yml up -d
-
-# Configure environment
-export DB_TYPE=victoriametrics
-export VM_HOST=localhost
-export VM_PORT=8428
-
-# Run monitor
-python water_scraper_v3.py
-```
-
-### **3. Test Different Databases**
-```bash
-# Test all supported databases
-python demo_databases.py all
-
-# Test specific database
-python demo_databases.py victoriametrics
-```
-
-## 📊 **Data Collection**
-
-### **Station Coverage**
-- **16 Water Monitoring Stations** across Thailand
-- **Accurate Station Codes**: P.1, P.20, P.21, P.4A, P.5, P.67, P.75, P.76, P.77, P.81, P.82, P.84, P.85, P.87, P.92, P.103
-- **Bilingual Names**: Thai and English station identification
-
-### **Metrics Collected**
-- 🌊 **Water Level**: Measured in meters (m)
-- 💧 **Discharge**: Measured in cubic meters per second (cms)
-- 📊 **Discharge Percentage**: Relative to station capacity
-- ⏰ **Timestamp**: Hour 24 handling (midnight = 00:00 next day)
-
-### **Data Frequency**
-- **Every 15 Minutes**: Continuous monitoring
-- **~300+ Data Points**: Per collection cycle
-- **Automatic Gap Filling**: Historical data recovery
-- **Data Updates**: Changed values detection and correction
-
-## 🔧 **Command Line Tools**
-
-### **Main Application**
-```bash
-python water_scraper_v3.py                    # Run continuous monitoring
-python water_scraper_v3.py --test             # Single test cycle
-python water_scraper_v3.py --help             # Show help
-```
-
-### **Gap Management**
-```bash
-python water_scraper_v3.py --check-gaps [days]  # Check for missing data
-python water_scraper_v3.py --fill-gaps [days]   # Fill missing data gaps
-python water_scraper_v3.py --update-data [days] # Update existing data
-```
-
-### **Database Testing**
-```bash
-python demo_databases.py                      # SQLite demo
-python demo_databases.py victoriametrics      # VictoriaMetrics demo
-python demo_databases.py all                  # Test all databases
-```
-
-## 📈 **Monitoring & Visualization**
-
-### **Grafana Dashboard**
-- **URL**: http://localhost:3000 (when using docker-compose)
-- **Username**: admin
-- **Password**: admin_password
-- **Features**: Time series charts, status tables, gauges, alerts
-
-### **VictoriaMetrics API**
-- **URL**: http://localhost:8428
-- **Health**: http://localhost:8428/health
-- **Metrics**: http://localhost:8428/metrics
-- **Query API**: http://localhost:8428/api/v1/query
-
-## 🛡️ **Security & Production**
-
-### **HTTPS Configuration**
-- Complete guide in `HTTPS_CONFIGURATION.md`
-- SSL certificate setup
-- Reverse proxy configuration
-- Security best practices
-
-### **Deployment Options**
-- **Docker**: Containerized deployment
-- **Systemd**: Linux service configuration
-- **Cloud**: AWS, GCP, Azure deployment guides
-- **Monitoring**: Health checks and alerting
-
-## 📚 **Documentation**
-
-### **Available Guides**
-1. **README.md** - Main project documentation
-2. **DATABASE_DEPLOYMENT_GUIDE.md** - Multi-database setup
-3. **ENHANCED_SCHEDULER_GUIDE.md** - 15-minute scheduler details
-4. **GAP_FILLING_GUIDE.md** - Data integrity and gap filling
-5. **DEBIAN_TROUBLESHOOTING.md** - Linux deployment troubleshooting
-6. **VICTORIAMETRICS_SETUP.md** - VictoriaMetrics configuration
-7. **HTTPS_CONFIGURATION.md** - Secure deployment setup
-
-### **Key Features Documented**
-- ✅ Installation and configuration
-- ✅ Multi-database support
-- ✅ 15-minute scheduling system
-- ✅ Gap filling and data integrity
-- ✅ Production deployment
-- ✅ Monitoring and troubleshooting
-- ✅ Security configuration
-
-## 🎯 **Project Status: PRODUCTION READY**
-
-The Thailand Water Monitor is now:
-- ✅ **Clean**: All old and redundant files removed
-- ✅ **Organized**: Clear project structure with proper documentation
-- ✅ **Enhanced**: 15-minute scheduling with gap filling
-- ✅ **Scalable**: Multi-database support with VictoriaMetrics
-- ✅ **Secure**: HTTPS configuration and security best practices
-- ✅ **Monitored**: Comprehensive logging and Grafana dashboards
-- ✅ **Documented**: Complete guides for all features and deployment options
-
-The project is ready for production deployment with professional-grade monitoring capabilities.

docs/PROJECT_STRUCTURE.md

@@ -1,272 +0,0 @@
-# 🏗️ Project Structure - Northern Thailand Ping River Monitor
-
-## 📁 Directory Layout
-
-```
-Northern-Thailand-Ping-River-Monitor/
-├── 📁 src/                          # Main application source code
-│   ├── __init__.py                  # Package initialization
-│   ├── main.py                      # CLI entry point and main application
-│   ├── water_scraper_v3.py          # Core data collection engine
-│   ├── web_api.py                   # FastAPI web interface
-│   ├── config.py                    # Configuration management
-│   ├── database_adapters.py         # Database abstraction layer
-│   ├── models.py                    # Data models and type definitions
-│   ├── exceptions.py                # Custom exception classes
-│   ├── validators.py                # Data validation layer
-│   ├── metrics.py                   # Metrics collection system
-│   ├── health_check.py              # Health monitoring system
-│   ├── rate_limiter.py              # Rate limiting and request tracking
-│   └── logging_config.py            # Enhanced logging configuration
-├── 📁 docs/                         # Documentation files
-│   ├── STATION_MANAGEMENT_GUIDE.md  # Station management documentation
-│   ├── ENHANCEMENT_SUMMARY.md       # Feature enhancement summary
-│   └── PROJECT_STRUCTURE.md         # This file
-├── 📁 scripts/                      # Utility scripts
-│   └── migrate_geolocation.py       # Database migration script
-├── 📁 grafana/                      # Grafana configuration
-│   ├── dashboards/                  # Dashboard definitions
-│   └── provisioning/                # Grafana provisioning config
-├── 📁 tests/                        # Test files
-│   ├── test_integration.py          # Integration test suite
-│   ├── test_station_management.py   # Station management tests
-│   └── test_api.py                  # API endpoint tests
-├── 📄 run.py                        # Simple startup script
-├── 📄 requirements.txt              # Production dependencies
-├── 📄 requirements-dev.txt          # Development dependencies
-├── 📄 setup.py                      # Package installation script
-├── 📄 Dockerfile                    # Docker container definition
-├── 📄 docker-compose.victoriametrics.yml  # Complete stack deployment
-├── 📄 Makefile                      # Common development tasks
-├── 📄 .env.example                  # Environment configuration template
-├── 📄 .gitignore                    # Git ignore patterns
-├── 📄 .gitlab-ci.yml                # CI/CD pipeline configuration
-├── 📄 LICENSE                       # MIT license
-├── 📄 README.md                     # Main project documentation
-└── 📄 CONTRIBUTING.md               # Contribution guidelines
-```
-
-## 🔧 Core Components
-
-### **Application Layer**
-- **`src/main.py`** - Command-line interface and application orchestration
-- **`src/web_api.py`** - FastAPI web interface with REST endpoints
-- **`src/water_scraper_v3.py`** - Core data collection and processing engine
-
-### **Data Layer**
-- **`src/database_adapters.py`** - Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics)
-- **`src/models.py`** - Pydantic data models and type definitions
-- **`src/validators.py`** - Data validation and sanitization
-
-### **Infrastructure Layer**
-- **`src/config.py`** - Configuration management with environment variable support
-- **`src/logging_config.py`** - Structured logging with rotation and colors
-- **`src/metrics.py`** - Application metrics collection (counters, gauges, histograms)
-- **`src/health_check.py`** - System health monitoring and status checks
-
-### **Utility Layer**
-- **`src/exceptions.py`** - Custom exception hierarchy
-- **`src/rate_limiter.py`** - API rate limiting and request tracking
-
-## 🌐 Web API Structure
-
-### **Endpoints Organization**
-```
-/                           # Dashboard homepage
-├── /health                 # System health status
-├── /metrics               # Application metrics
-├── /config                # Configuration (masked)
-├── /stations              # Station management
-│   ├── GET /              # List all stations
-│   ├── POST /             # Create new station
-│   ├── GET /{id}          # Get specific station
-│   ├── PUT /{id}          # Update station
-│   └── DELETE /{id}       # Delete station
-├── /measurements          # Data access
-│   ├── /latest            # Latest measurements
-│   └── /station/{code}    # Station-specific data
-└── /scraping              # Data collection control
-    ├── /trigger           # Manual data collection
-    └── /status            # Scraping status
-```
-
-### **API Models**
-- **Request Models**: Station creation/update, query parameters
-- **Response Models**: Station info, measurements, health status
-- **Error Models**: Standardized error responses
-
-## 🗄️ Database Architecture
-
-### **Supported Databases**
-1. **SQLite** - Local development and testing
-2. **MySQL** - Traditional relational database
-3. **PostgreSQL** - Advanced relational with TimescaleDB support
-4. **InfluxDB** - Purpose-built time-series database
-5. **VictoriaMetrics** - High-performance metrics storage
-
-### **Schema Design**
-```sql
--- Stations table
-stations (
-    id INTEGER PRIMARY KEY,
-    station_code VARCHAR(10) UNIQUE,
-    thai_name VARCHAR(255),
-    english_name VARCHAR(255),
-    latitude DECIMAL(10,8),
-    longitude DECIMAL(11,8),
-    geohash VARCHAR(20),
-    status VARCHAR(20),
-    created_at TIMESTAMP,
-    updated_at TIMESTAMP
-)
-
--- Measurements table
-water_measurements (
-    id BIGINT PRIMARY KEY,
-    timestamp DATETIME,
-    station_id INTEGER,
-    water_level DECIMAL(10,3),
-    discharge DECIMAL(10,2),
-    discharge_percent DECIMAL(5,2),
-    status VARCHAR(20),
-    created_at TIMESTAMP,
-    FOREIGN KEY (station_id) REFERENCES stations(id),
-    UNIQUE(timestamp, station_id)
-)
-```
-
-## 🐳 Docker Architecture
-
-### **Multi-Stage Build**
-1. **Builder Stage** - Compile dependencies and build artifacts
-2. **Production Stage** - Minimal runtime environment
-
-### **Service Composition**
-- **ping-river-monitor** - Data collection service
-- **ping-river-api** - Web API service
-- **victoriametrics** - Time-series database
-- **grafana** - Visualization dashboard
-
-## 📊 Monitoring Architecture
-
-### **Metrics Collection**
-- **Counters** - API requests, database operations, scraping cycles
-- **Gauges** - Current values, connection status, resource usage
-- **Histograms** - Response times, processing durations
-
-### **Health Checks**
-- **Database Health** - Connection status, data freshness
-- **API Health** - External API availability, response times
-- **System Health** - Memory usage, disk space, CPU load
-
-### **Logging Levels**
-- **DEBUG** - Detailed execution information
-- **INFO** - General operational messages
-- **WARNING** - Potential issues and recoverable errors
-- **ERROR** - Serious problems requiring attention
-- **CRITICAL** - System-threatening issues
-
-## 🔧 Configuration Management
-
-### **Environment Variables**
-```bash
-# Database
-DB_TYPE=victoriametrics
-VM_HOST=localhost
-VM_PORT=8428
-
-# Application
-SCRAPING_INTERVAL_HOURS=1
-LOG_LEVEL=INFO
-DATA_RETENTION_DAYS=365
-
-# Security
-SECRET_KEY=your-secret-key
-API_KEY=your-api-key
-```
-
-### **Configuration Hierarchy**
-1. Environment variables (highest priority)
-2. .env file
-3. Default values in config.py (lowest priority)
-
-## 🧪 Testing Architecture
-
-### **Test Categories**
-- **Unit Tests** - Individual component testing
-- **Integration Tests** - System component interaction
-- **API Tests** - Endpoint functionality and responses
-- **Performance Tests** - Load and stress testing
-
-### **Test Data**
-- **Mock Data** - Simulated API responses
-- **Test Database** - Isolated test environment
-- **Fixtures** - Reusable test data sets
-
-## 📦 Deployment Architecture
-
-### **Development**
-```bash
-python run.py --web-api    # Local development server
-```
-
-### **Production**
-```bash
-docker-compose up -d       # Full stack deployment
-```
-
-### **CI/CD Pipeline**
-1. **Test Stage** - Run all tests and quality checks
-2. **Build Stage** - Create Docker images
-3. **Deploy Stage** - Deploy to staging/production
-4. **Health Check** - Verify deployment success
-
-## 🔒 Security Architecture
-
-### **Input Validation**
-- Pydantic models for API requests
-- Data range validation for measurements
-- SQL injection prevention through ORM
-
-### **Authentication** (Future)
-- API key authentication
-- JWT token support
-- Role-based access control
-
-### **Data Protection**
-- Environment variable configuration
-- Sensitive data masking in logs
-- HTTPS support for production
-
-## 📈 Performance Architecture
-
-### **Optimization Strategies**
-- Database connection pooling
-- Query optimization and indexing
-- Response caching for static data
-- Async processing for I/O operations
-
-### **Scalability Considerations**
-- Horizontal scaling with load balancers
-- Database read replicas
-- Microservice architecture readiness
-- Container orchestration support
-
-## 🔄 Data Flow Architecture
-
-### **Collection Flow**
-```
-External API → Rate Limiter → Data Validator → Database Adapter → Database
-```
-
-### **API Flow**
-```
-HTTP Request → FastAPI → Business Logic → Database Adapter → HTTP Response
-```
-
-### **Monitoring Flow**
-```
-Application Events → Metrics Collector → Health Checks → Monitoring Dashboard
-```
-
-This architecture provides a solid foundation for a production-ready water monitoring system with excellent maintainability, scalability, and observability.