Implement adaptive scheduler with intelligent retry logic

- Replace fixed hourly schedule with adaptive scheduling system
- Switch to 1-minute retries when no data is available from API
- Return to hourly schedule once data is successfully fetched
- Fix data fetching to use yesterday's date (API has 1-day delay)
- Add comprehensive logging for scheduler mode changes
- Improve resilience against API data availability issues

The scheduler now intelligently adapts to data availability:
- Normal mode: hourly runs at top of each hour
- Retry mode: minute-based retries until data is available
- Automatic mode switching based on fetch success/failure

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2025-09-28 18:24:33 +07:00
co-authored by Claude
parent 0ff58ecb13
commit 4cc792157f
2 changed files with 52 additions and 24 deletions
+3 -3
View File
@@ -407,9 +407,9 @@ class EnhancedWaterMonitorScraper:
return None
def fetch_water_data(self) -> Optional[List[Dict]]:
"""Fetch water levels and discharge data from API for current date"""
current_date = datetime.datetime.now()
return self.fetch_water_data_for_date(current_date)
"""Fetch water levels and discharge data from API for yesterday (API has 1-day delay)"""
yesterday = datetime.datetime.now() - datetime.timedelta(days=1)
return self.fetch_water_data_for_date(yesterday)
def save_to_database(self, water_data: List[Dict], max_retries: int = 3) -> bool:
"""Save water measurements to database with retry logic"""