Integrate automatic alerting into continuous monitoring
Some checks failed
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.10) (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.12) (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.9) (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Has been cancelled
Security & Dependency Updates / Dependency Security Scan (push) Has been cancelled
Security & Dependency Updates / License Compliance (push) Has been cancelled
Security & Dependency Updates / Check for Dependency Updates (push) Has been cancelled
Security & Dependency Updates / Code Quality Metrics (push) Has been cancelled
Security & Dependency Updates / Security Summary (push) Has been cancelled
Some checks failed
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.10) (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.12) (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.9) (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Has been cancelled
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Has been cancelled
Security & Dependency Updates / Dependency Security Scan (push) Has been cancelled
Security & Dependency Updates / License Compliance (push) Has been cancelled
Security & Dependency Updates / Check for Dependency Updates (push) Has been cancelled
Security & Dependency Updates / Code Quality Metrics (push) Has been cancelled
Security & Dependency Updates / Security Summary (push) Has been cancelled
- Alerts now run automatically after every successful new data fetch - Works for both hourly fetches and retry mode exits - Alert check runs when fresh data is saved to database - Logs alert results (total generated and sent count) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
17
src/main.py
17
src/main.py
@@ -64,7 +64,7 @@ def run_test_cycle():
|
||||
return False
|
||||
|
||||
def run_continuous_monitoring():
|
||||
"""Run continuous monitoring with adaptive scheduling"""
|
||||
"""Run continuous monitoring with adaptive scheduling and alerting"""
|
||||
logger.info("Starting continuous monitoring...")
|
||||
|
||||
try:
|
||||
@@ -75,11 +75,16 @@ def run_continuous_monitoring():
|
||||
db_config = Config.get_database_config()
|
||||
scraper = EnhancedWaterMonitorScraper(db_config)
|
||||
|
||||
# Initialize alerting system
|
||||
from .alerting import WaterLevelAlertSystem
|
||||
alerting = WaterLevelAlertSystem()
|
||||
|
||||
# Setup signal handlers
|
||||
setup_signal_handlers(scraper)
|
||||
|
||||
logger.info(f"Monitoring started with {Config.SCRAPING_INTERVAL_HOURS}h interval")
|
||||
logger.info("Adaptive retry: switches to 1-minute intervals when no data available")
|
||||
logger.info("Alerts: automatic check after each successful data fetch")
|
||||
logger.info("Press Ctrl+C to stop")
|
||||
|
||||
# Run initial cycle
|
||||
@@ -109,6 +114,16 @@ def run_continuous_monitoring():
|
||||
|
||||
if success:
|
||||
last_successful_fetch = current_time
|
||||
|
||||
# Run alert check after every successful new data fetch
|
||||
logger.info("Running alert check...")
|
||||
try:
|
||||
alert_results = alerting.run_alert_check()
|
||||
if alert_results.get('total_alerts', 0) > 0:
|
||||
logger.info(f"Alerts: {alert_results['total_alerts']} generated, {alert_results['sent']} sent")
|
||||
except Exception as e:
|
||||
logger.error(f"Alert check failed: {e}")
|
||||
|
||||
if retry_mode:
|
||||
logger.info("✅ Data fetch successful - switching back to hourly schedule")
|
||||
retry_mode = False
|
||||
|
Reference in New Issue
Block a user