perf: stale-while-revalidate caching, bigger executor, cheap DB health probe
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 16s
Documentation / Build Sphinx Documentation (push) Successful in 15s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 0s
Documentation / Documentation Summary (push) Successful in 2s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 25s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
Documentation / Validate Documentation (push) Failing after 13s
Documentation / Generate API Documentation (push) Successful in 11s

Third on-box run: p50s healthy everywhere but tails at 60s — when a TTL
expired under load, every concurrent miss parked an executor thread on
the single-flight lock, exhausting the ~12-thread pool and timing out
unrelated endpoints. All cached endpoints (latest, HII, stats, health)
now use _cached_swr: fresh -> inline; expired-but-present -> the stale
value is returned immediately and ONE background task refreshes; only a
cold key (first request since startup) waits. Plus: dedicated
ThreadPoolExecutor (EXECUTOR_THREADS, default 48) replaces the cpu+4
default, and DatabaseHealthCheck no longer re-runs the CREATE TABLE DDL
suite on every probe (connect only when no live engine).
This commit is contained in:
2026-08-12 12:32:38 +07:00
parent 96fedb3991
commit 731f10910e
4 changed files with 97 additions and 50 deletions
+5
View File
@@ -94,6 +94,11 @@ class Config:
# TTL for /health check results (includes an external RID-API probe)
HEALTH_CACHE_TTL_SECONDS = int(os.getenv("HEALTH_CACHE_TTL_SECONDS", "30"))
# Thread-pool size for blocking work in the web process (DB queries,
# inference, health probes). Waiting threads are cheap; starving the pool
# stalls every endpoint that needs a thread.
EXECUTOR_THREADS = int(os.getenv("EXECUTOR_THREADS", "48"))
# Web server worker processes. Above 1, uvicorn forks workers and a
# localhost lock port elects a single background-collection leader.
WEB_WORKERS = int(os.getenv("WEB_WORKERS", "2"))