fix: protect admin API endpoints; stop rejecting flood-peak measurements
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 25s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 12s
Documentation / Validate Documentation (push) Failing after 9s
Documentation / Generate API Documentation (push) Successful in 10s
Documentation / Build Sphinx Documentation (push) Successful in 16s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 0s
Documentation / Documentation Summary (push) Successful in 2s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Test Suite (3.11) (push) Failing after 25s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Build Docker Image (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Integration Test with Services (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Staging (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Deploy to Production (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Performance Test (push) Skipped
CI/CD Pipeline - Northern Thailand Ping River Monitor / Code Quality (push) Successful in 12s
Documentation / Validate Documentation (push) Failing after 9s
Documentation / Generate API Documentation (push) Successful in 10s
Documentation / Build Sphinx Documentation (push) Successful in 16s
CI/CD Pipeline - Northern Thailand Ping River Monitor / Cleanup (push) Successful in 0s
Documentation / Documentation Summary (push) Successful in 2s
Security: POST/PUT/DELETE /stations, POST /scrape/trigger and GET /config now require an X-API-Key header matching ADMIN_API_KEY. Secure by default - with no key configured those endpoints return 503 instead of being open. Comparison via secrets.compare_digest. Dashboard and read endpoints stay public. Data: the validator rejected any measurement whose discharge_percent exceeded 200 - which silently deleted the Oct 2024 record-flood peaks (the river genuinely ran at 201-226% of channel capacity). The cap is now 500%, and an out-of-range percent nulls that auxiliary field instead of discarding the whole row (water level and discharge are the data that matter). Surfaced by the user's historical backfill log.
This commit is contained in:
+10
-4
@@ -22,7 +22,9 @@ class DataValidator:
|
||||
DISCHARGE_MIN = 0.0 # cms
|
||||
DISCHARGE_MAX = 10000.0 # cms
|
||||
DISCHARGE_PERCENT_MIN = 0.0
|
||||
DISCHARGE_PERCENT_MAX = 200.0 # Allow some overflow
|
||||
# % of channel capacity. Major floods genuinely exceed 200% (Oct 2024 peaked
|
||||
# at 226%); values beyond 500% are treated as data errors.
|
||||
DISCHARGE_PERCENT_MAX = 500.0
|
||||
|
||||
@classmethod
|
||||
def validate_measurement(cls, measurement: Dict[str, Any]) -> bool:
|
||||
@@ -59,7 +61,10 @@ class DataValidator:
|
||||
logger.warning(f"Discharge out of range: {discharge}")
|
||||
return False
|
||||
|
||||
# Validate discharge percent if present
|
||||
# Validate discharge percent if present. This is an auxiliary field:
|
||||
# an implausible value must not cost us the water level and discharge
|
||||
# (rejecting rows here silently deleted the Oct 2024 flood peaks), so
|
||||
# out-of-range percents are nulled and the measurement is kept.
|
||||
if measurement.get("discharge_percent") is not None:
|
||||
discharge_percent = float(measurement["discharge_percent"])
|
||||
if not (
|
||||
@@ -68,9 +73,10 @@ class DataValidator:
|
||||
<= cls.DISCHARGE_PERCENT_MAX
|
||||
):
|
||||
logger.warning(
|
||||
f"Discharge percent out of range: {discharge_percent}"
|
||||
f"Discharge percent out of range ({discharge_percent}); "
|
||||
"keeping measurement with discharge_percent=None"
|
||||
)
|
||||
return False
|
||||
measurement["discharge_percent"] = None
|
||||
|
||||
# Validate station ID
|
||||
station_id = measurement["station_id"]
|
||||
|
||||
Reference in New Issue
Block a user