Initial commit: Northern Thailand Ping River Monitor v3.1.0

Features: - Real-time water level monitoring for Ping River Basin (16 stations) - Coverage from Chiang Dao to Nakhon Sawan in Northern Thailand - FastAPI web interface with interactive dashboard and station management - Multi-database support (SQLite, MySQL, PostgreSQL, InfluxDB, VictoriaMetrics) - Comprehensive monitoring with health checks and metrics collection - Docker deployment with Grafana integration - Production-ready architecture with enterprise-grade observability CI/CD & Automation: - Complete Gitea Actions workflows for CI/CD, security, and releases - Multi-Python version testing (3.9-3.12) - Multi-architecture Docker builds (amd64, arm64) - Daily security scanning and dependency monitoring - Automated documentation generation - Performance testing and validation Production Ready: - Type safety with Pydantic models and comprehensive type hints - Data validation layer with range checking and error handling - Rate limiting and request tracking for API protection - Enhanced logging with rotation, colors, and performance metrics - Station management API for dynamic CRUD operations - Comprehensive documentation and deployment guides Technical Stack: - Python 3.9+ with FastAPI and Pydantic - Multi-database architecture with adapter pattern - Docker containerization with multi-stage builds - Grafana dashboards for visualization - Gitea Actions for CI/CD automation - Enterprise monitoring and alerting Ready for deployment to B4L infrastructure!
2025-08-12 15:37:09 +07:00
commit af62cfef0b
60 changed files with 13267 additions and 0 deletions
--- a/docs/GITEA_WORKFLOWS.md
+++ b/docs/GITEA_WORKFLOWS.md
@@ -0,0 +1,295 @@
+# 🔄 Gitea Actions Workflows - Northern Thailand Ping River Monitor
+
+## 📋 Overview
+
+This document describes the Gitea Actions workflows configured for the Northern Thailand Ping River Monitor project. These workflows provide comprehensive CI/CD, security scanning, and documentation generation.
+
+## 🚀 Available Workflows
+
+### 1. **CI/CD Pipeline** (`.gitea/workflows/ci.yml`)
+
+**Triggers:**
+- Push to `main` or `develop` branches
+- Pull requests to `main`
+- Daily scheduled runs at 2 AM UTC
+
+**Jobs:**
+- **Test Suite**: Multi-version Python testing (3.9-3.12)
+- **Code Quality**: Linting, formatting, and type checking
+- **Build**: Docker image creation and testing
+- **Integration Test**: Testing with VictoriaMetrics service
+- **Deploy Staging**: Automatic deployment to staging (develop branch)
+- **Deploy Production**: Manual deployment to production (main branch)
+- **Performance Test**: Load testing after production deployment
+
+**Key Features:**
+- ✅ Multi-Python version testing
+- ✅ Docker multi-architecture builds (amd64, arm64)
+- ✅ Service integration testing
+- ✅ Automatic staging deployment
+- ✅ Manual production approval
+- ✅ Performance validation
+
+### 2. **Security & Dependency Updates** (`.gitea/workflows/security.yml`)
+
+**Triggers:**
+- Daily scheduled runs at 3 AM UTC
+- Manual dispatch
+- Changes to requirements files or Dockerfile
+
+**Jobs:**
+- **Dependency Scan**: Safety, Bandit, Semgrep security scans
+- **Docker Security**: Trivy vulnerability scanning
+- **License Check**: License compliance verification
+- **Dependency Updates**: Automated update detection
+- **Code Quality**: Complexity and maintainability analysis
+
+**Key Features:**
+- 🔒 Daily security scans
+- 📦 Dependency vulnerability detection
+- 📄 License compliance checking
+- 🔄 Automated update notifications
+- 📊 Code quality metrics
+
+### 3. **Release Workflow** (`.gitea/workflows/release.yml`)
+
+**Triggers:**
+- Git tags matching `v*.*.*` pattern
+- Manual dispatch with version input
+
+**Jobs:**
+- **Create Release**: Automated release creation with changelog
+- **Test Release**: Comprehensive testing across Python versions
+- **Build Release**: Multi-architecture Docker images with proper tags
+- **Security Scan**: Trivy security scanning of release images
+- **Deploy Release**: Production deployment with health checks
+- **Validate Release**: Post-deployment validation and testing
+
+**Key Features:**
+- 🏷️ Automated release creation
+- 📝 Changelog generation
+- 🐳 Multi-architecture Docker builds
+- 🔒 Security scanning
+- ✅ Comprehensive validation
+
+### 4. **Documentation** (`.gitea/workflows/docs.yml`)
+
+**Triggers:**
+- Changes to documentation files
+- Changes to Python source files
+- Manual dispatch
+
+**Jobs:**
+- **Validate Docs**: Link checking and structure validation
+- **Generate API Docs**: OpenAPI specification generation
+- **Build Sphinx Docs**: Comprehensive API documentation
+- **Documentation Summary**: Build status and artifact summary
+
+**Key Features:**
+- 📚 Automated API documentation
+- 🔗 Link validation
+- 📖 Sphinx documentation generation
+- ✅ Documentation completeness checking
+
+## 🔧 Workflow Configuration
+
+### **Required Secrets**
+
+Configure these secrets in your Gitea repository settings:
+
+```bash
+GITEA_TOKEN          # Gitea access token for container registry
+SLACK_WEBHOOK_URL    # Optional: Slack notifications
+STAGING_WEBHOOK_URL  # Optional: Staging deployment webhook
+PRODUCTION_WEBHOOK_URL # Optional: Production deployment webhook
+```
+
+### **Environment Variables**
+
+Key environment variables used across workflows:
+
+```yaml
+PYTHON_VERSION: '3.11'           # Default Python version
+REGISTRY: git.b4l.co.th          # Container registry
+IMAGE_NAME: grabowski/northern-thailand-ping-river-monitor
+```
+
+## 📊 Workflow Status
+
+### **CI/CD Pipeline Status**
+- **Test Coverage**: Multi-version Python testing
+- **Code Quality**: Automated linting and formatting
+- **Security**: Integrated security scanning
+- **Deployment**: Automated staging, manual production
+
+### **Security Monitoring**
+- **Daily Scans**: Automated vulnerability detection
+- **Dependency Updates**: Proactive update notifications
+- **License Compliance**: Automated license checking
+- **Code Quality**: Continuous quality monitoring
+
+### **Release Management**
+- **Automated Releases**: Tag-based release creation
+- **Multi-Architecture**: Support for amd64 and arm64
+- **Security Validation**: Pre-deployment security checks
+- **Health Monitoring**: Post-deployment validation
+
+## 🚀 Usage Examples
+
+### **Triggering Workflows**
+
+**Manual CI/CD Run:**
+```bash
+# Push to trigger CI/CD
+git push origin main
+
+# Create pull request to trigger testing
+git checkout -b feature/new-feature
+git push origin feature/new-feature
+# Create PR in Gitea UI
+```
+
+**Manual Security Scan:**
+```bash
+# Trigger via Gitea Actions UI
+# Go to Actions → Security & Dependency Updates → Run workflow
+```
+
+**Creating a Release:**
+```bash
+# Create and push a tag
+git tag v3.1.1
+git push origin v3.1.1
+
+# Or use manual dispatch in Gitea Actions UI
+```
+
+### **Monitoring Workflow Results**
+
+**Check Workflow Status:**
+1. Navigate to your repository in Gitea
+2. Click on "Actions" tab
+3. View workflow runs and their status
+
+**Download Artifacts:**
+1. Click on a completed workflow run
+2. Scroll to "Artifacts" section
+3. Download reports and logs
+
+**View Security Reports:**
+1. Go to Security workflow runs
+2. Download security-reports artifacts
+3. Review JSON reports for vulnerabilities
+
+## 🔍 Troubleshooting
+
+### **Common Issues**
+
+**Workflow Fails on Dependencies:**
+```bash
+# Check requirements.txt for version conflicts
+pip-compile requirements.in
+```
+
+**Docker Build Fails:**
+```bash
+# Test Docker build locally
+make docker-build
+docker run --rm ping-river-monitor python run.py --test
+```
+
+**Security Scan Failures:**
+```bash
+# Run security scans locally
+safety check -r requirements.txt
+bandit -r src/
+```
+
+**Test Failures:**
+```bash
+# Run tests locally
+make test
+python tests/test_integration.py
+```
+
+### **Debugging Workflows**
+
+**Enable Debug Logging:**
+Add to workflow file:
+```yaml
+env:
+  ACTIONS_STEP_DEBUG: true
+  ACTIONS_RUNNER_DEBUG: true
+```
+
+**Check Workflow Logs:**
+1. Go to failed workflow run
+2. Click on failed job
+3. Expand failed step to see detailed logs
+
+**Validate Workflow Syntax:**
+```bash
+# Validate YAML syntax
+make validate-workflows
+```
+
+## 📈 Performance Optimization
+
+### **Caching Strategy**
+- **Pip Cache**: Cached across workflow runs
+- **Docker Layer Cache**: GitHub Actions cache for faster builds
+- **Dependency Cache**: Cached based on requirements.txt hash
+
+### **Parallel Execution**
+- **Matrix Builds**: Multiple Python versions tested in parallel
+- **Independent Jobs**: Security scans run independently of tests
+- **Conditional Execution**: Jobs skip when not needed
+
+### **Resource Management**
+- **Timeout Settings**: Prevent hanging workflows
+- **Resource Limits**: Appropriate runner sizing
+- **Artifact Cleanup**: Automatic cleanup of old artifacts
+
+## 🔒 Security Best Practices
+
+### **Secret Management**
+- Use Gitea repository secrets for sensitive data
+- Never commit secrets to repository
+- Rotate secrets regularly
+- Use least-privilege access tokens
+
+### **Container Security**
+- Multi-stage Docker builds for smaller images
+- Non-root user in containers
+- Regular base image updates
+- Vulnerability scanning before deployment
+
+### **Code Security**
+- Automated security scanning in CI/CD
+- Dependency vulnerability monitoring
+- License compliance checking
+- Code quality enforcement
+
+## 📚 Additional Resources
+
+### **Gitea Actions Documentation**
+- [Gitea Actions Overview](https://docs.gitea.io/en-us/usage/actions/)
+- [Workflow Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions)
+- [Available Actions](https://github.com/marketplace?type=actions)
+
+### **Project-Specific Resources**
+- [Contributing Guide](../CONTRIBUTING.md)
+- [Deployment Checklist](../DEPLOYMENT_CHECKLIST.md)
+- [Project Structure](PROJECT_STRUCTURE.md)
+
+### **Monitoring and Alerts**
+- Workflow status badges in README
+- Email notifications for failures
+- Slack/Discord integration for team updates
+- Grafana dashboards for deployment metrics
+
+---
+
+**Workflow Version**: v3.1.0  
+**Last Updated**: 2025-08-12  
+**Maintained By**: Ping River Monitor Team