grabowski
af62cfef0b
Some checks failed
Security & Dependency Updates / Dependency Security Scan (push) Successful in 29s
Security & Dependency Updates / Docker Security Scan (push) Failing after 53s
Security & Dependency Updates / License Compliance (push) Successful in 13s
Security & Dependency Updates / Check for Dependency Updates (push) Successful in 19s
Security & Dependency Updates / Code Quality Metrics (push) Successful in 11s
Security & Dependency Updates / Security Summary (push) Successful in 7s
Features: - Real-time water level monitoring for Ping River Basin (16 stations) - Coverage from Chiang Dao to Nakhon Sawan in Northern Thailand - FastAPI web interface with interactive dashboard and station management - Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics) - Comprehensive monitoring with health checks and metrics collection - Docker deployment with Grafana integration - Production-ready architecture with enterprise-grade observability CI/CD & Automation: - Complete Gitea Actions workflows for CI/CD, security, and releases - Multi-Python version testing (3.9-3.12) - Multi-architecture Docker builds (amd64, arm64) - Daily security scanning and dependency monitoring - Automated documentation generation - Performance testing and validation Production Ready: - Type safety with Pydantic models and comprehensive type hints - Data validation layer with range checking and error handling - Rate limiting and request tracking for API protection - Enhanced logging with rotation, colors, and performance metrics - Station management API for dynamic CRUD operations - Comprehensive documentation and deployment guides Technical Stack: - Python 3.9+ with FastAPI and Pydantic - Multi-database architecture with adapter pattern - Docker containerization with multi-stage builds - Grafana dashboards for visualization - Gitea Actions for CI/CD automation - Enterprise monitoring and alerting Ready for deployment to B4L infrastructure!
137 lines
3.3 KiB
Markdown
137 lines
3.3 KiB
Markdown
# Geolocation Migration Quick Start
|
|
|
|
This is a quick reference guide for updating a running Thailand Water Monitor system to add geolocation support for Grafana geomap.
|
|
|
|
## 🚀 **Quick Migration (5 minutes)**
|
|
|
|
### **Step 1: Stop Application**
|
|
```bash
|
|
# Stop the service (choose your method)
|
|
sudo systemctl stop water-monitor
|
|
# OR
|
|
docker stop water-monitor
|
|
# OR use Ctrl+C if running manually
|
|
```
|
|
|
|
### **Step 2: Backup Database**
|
|
```bash
|
|
# SQLite backup
|
|
cp water_monitoring.db water_monitoring.db.backup
|
|
|
|
# PostgreSQL backup
|
|
pg_dump water_monitoring > backup.sql
|
|
|
|
# MySQL backup
|
|
mysqldump water_monitoring > backup.sql
|
|
```
|
|
|
|
### **Step 3: Run Migration**
|
|
```bash
|
|
# Run the automated migration script
|
|
python migrate_geolocation.py
|
|
```
|
|
|
|
### **Step 4: Restart Application**
|
|
```bash
|
|
# Restart the service
|
|
sudo systemctl start water-monitor
|
|
# OR
|
|
docker start water-monitor
|
|
# OR
|
|
python water_scraper_v3.py
|
|
```
|
|
|
|
## ✅ **Expected Output**
|
|
```
|
|
2025-07-28 17:30:00,123 - INFO - Starting geolocation column migration...
|
|
2025-07-28 17:30:00,124 - INFO - Detected database type: SQLITE
|
|
2025-07-28 17:30:00,127 - INFO - Added latitude column
|
|
2025-07-28 17:30:00,128 - INFO - Added longitude column
|
|
2025-07-28 17:30:00,129 - INFO - Added geohash column
|
|
2025-07-28 17:30:00,133 - INFO - ✅ Migration completed successfully!
|
|
```
|
|
|
|
## 🗺️ **Verify Geolocation Works**
|
|
|
|
### **Check Database**
|
|
```bash
|
|
# SQLite
|
|
sqlite3 water_monitoring.db "SELECT station_code, latitude, longitude, geohash FROM stations WHERE station_code = 'P.1';"
|
|
|
|
# Expected output: P.1|15.6944|100.2028|w5q6uuhvfcfp25
|
|
```
|
|
|
|
### **Test Application**
|
|
```bash
|
|
# Run a test cycle
|
|
python water_scraper_v3.py --test
|
|
|
|
# Should complete without errors
|
|
```
|
|
|
|
## 🔧 **Grafana Setup**
|
|
|
|
### **Query for Geomap**
|
|
```sql
|
|
SELECT
|
|
s.latitude, s.longitude, s.station_code, s.english_name,
|
|
m.water_level, m.discharge_percent
|
|
FROM stations s
|
|
JOIN water_measurements m ON s.id = m.station_id
|
|
WHERE s.latitude IS NOT NULL
|
|
AND m.timestamp = (SELECT MAX(timestamp) FROM water_measurements WHERE station_id = s.id)
|
|
```
|
|
|
|
### **Geomap Configuration**
|
|
1. Create new panel → Select "Geomap"
|
|
2. Set **Latitude field**: `latitude`
|
|
3. Set **Longitude field**: `longitude`
|
|
4. Set **Color field**: `water_level`
|
|
5. Set **Size field**: `discharge_percent`
|
|
|
|
## 🚨 **Troubleshooting**
|
|
|
|
### **Database Locked**
|
|
```bash
|
|
sudo systemctl stop water-monitor
|
|
pkill -f water_scraper
|
|
sleep 5
|
|
python migrate_geolocation.py
|
|
```
|
|
|
|
### **Permission Error**
|
|
```bash
|
|
sudo chown $USER:$USER water_monitoring.db
|
|
chmod 664 water_monitoring.db
|
|
```
|
|
|
|
### **Missing Dependencies**
|
|
```bash
|
|
pip install psycopg2-binary pymysql
|
|
```
|
|
|
|
## 🔄 **Rollback (if needed)**
|
|
```bash
|
|
# Stop application
|
|
sudo systemctl stop water-monitor
|
|
|
|
# Restore backup
|
|
cp water_monitoring.db.backup water_monitoring.db
|
|
|
|
# Restart
|
|
sudo systemctl start water-monitor
|
|
```
|
|
|
|
## 📚 **More Information**
|
|
- **Full Guide**: See `GEOLOCATION_GUIDE.md`
|
|
- **Migration Script**: `migrate_geolocation.py`
|
|
- **Database Schema**: Updated with latitude, longitude, geohash columns
|
|
|
|
## 🎯 **What You Get**
|
|
- ✅ **P.1 Station** ready for geomap (Nawarat Bridge)
|
|
- ✅ **Database Schema** updated for all 16 stations
|
|
- ✅ **Grafana Compatible** data structure
|
|
- ✅ **Backward Compatible** - existing data preserved
|
|
|
|
**Total Time**: ~5 minutes for complete migration
|