feat: --fill-gaps all repairs missing data across the whole DB range
Documentation / Generate API Documentation (push) Successful in 11s
Documentation / Build Sphinx Documentation (push) Successful in 17s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 1s
Documentation / Documentation Summary (push) Successful in 3s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 29s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 14s
Documentation / Validate Documentation (push) Failing after 9s

Gap detection is now hour-granular: days with partial data (missing
hourly slots) are re-fetched, not just days with no rows at all. The
API's hour-24-is-next-midnight quirk is handled by re-fetching day D-1
when day D is missing its 00:00 slot. SQL adapters gain single-query
range and recorded-hours lookups; non-SQL backends fall back to the
old day-granular check.
This commit is contained in:
2026-08-10 22:13:58 +07:00
parent 410faeddd5
commit 5e62ea529d
3 changed files with 236 additions and 37 deletions
+23 -7
View File
@@ -180,9 +180,12 @@ def run_continuous_monitoring():
return True
def run_gap_filling(days_back: int):
"""Run gap filling for missing data"""
logger.info(f"Checking for data gaps in the last {days_back} days...")
def run_gap_filling(days_back: Optional[int]):
"""Run gap filling for missing data (days_back=None scans the whole range)"""
if days_back is None:
logger.info("Checking for data gaps across the whole data range...")
else:
logger.info(f"Checking for data gaps in the last {days_back} days...")
try:
# Validate configuration
@@ -445,6 +448,7 @@ Examples:
%(prog)s # Run continuous monitoring
%(prog)s --web-api # Start web API server
%(prog)s --fill-gaps 7 # Fill missing data for last 7 days
%(prog)s --fill-gaps all # Fill missing data across the whole data range
%(prog)s --update-data 2 # Update existing data for last 2 days
%(prog)s --import-historical 2024-01-01 2024-01-31 # Import historical data
%(prog)s --status # Show system status
@@ -461,9 +465,11 @@ Examples:
parser.add_argument(
"--fill-gaps",
type=int,
metavar="DAYS",
help="Fill missing data gaps for the specified number of days back",
metavar="DAYS|all",
help=(
"Fill missing data gaps for the specified number of days back, "
"or 'all' to scan the entire data range in the database"
),
)
parser.add_argument(
@@ -529,7 +535,17 @@ Examples:
elif args.web_api:
success = run_web_api()
elif args.fill_gaps is not None:
success = run_gap_filling(args.fill_gaps)
if args.fill_gaps.lower() == "all":
success = run_gap_filling(None)
else:
try:
success = run_gap_filling(int(args.fill_gaps))
except ValueError:
logger.error(
f"Invalid --fill-gaps value '{args.fill_gaps}': "
"expected a number of days or 'all'"
)
sys.exit(1)
elif args.update_data is not None:
success = run_data_update(args.update_data)
elif args.import_historical is not None: