From c86266fd389dd74cd05e9be331e7ba58ad46d783 Mon Sep 17 00:00:00 2001 From: grabowski Date: Mon, 11 Aug 2025 10:19:55 +0700 Subject: [PATCH] Add CI auto-sort system for datasheets and images - Add DS_ prefix for datasheets - automatically moved to docs/datasheets/ - Add IMG_ prefix for images - automatically moved to docs/images/ - Update FILE-NAMING-AND-VERSIONING.md with CI auto-sort prefix examples - Create comprehensive docs/datasheets/README.md with CI auto-sort documentation - Create comprehensive docs/images/README.md with CI auto-sort documentation - Add ci/auto-sort-files.yml GitHub Actions workflow for automatic file organization - Update ci/README.md with complete auto-sort system documentation - Update main README.md to reference CI auto-sort system - Enable users to upload files anywhere with correct prefix for automatic organization - Support multiple file types: PDF, DOC, PNG, JPG, SVG, etc. - Provide complete CI workflow with commit automation and error handling --- FILE-NAMING-AND-VERSIONING.md | 14 +++ README.md | 3 +- ci/README.md | 160 +++++++++++++++++++++++++++++++++- ci/auto-sort-files.yml | 113 ++++++++++++++++++++++++ docs/datasheets/README.md | 89 ++++++++++++++++++- docs/images/README.md | 153 +++++++++++++++++++++++++++++++- 6 files changed, 528 insertions(+), 4 deletions(-) create mode 100644 ci/auto-sort-files.yml diff --git a/FILE-NAMING-AND-VERSIONING.md b/FILE-NAMING-AND-VERSIONING.md index cac9945..1a3592e 100644 --- a/FILE-NAMING-AND-VERSIONING.md +++ b/FILE-NAMING-AND-VERSIONING.md @@ -46,6 +46,20 @@ my-project_bom_v1.3.0_2024-08-04.csv my-project_schematic-plot_v1.0.0_2024-08-04.pdf ``` +#### Datasheets and Images (CI Auto-Sort) +For automatic organization by CI bots, use these prefixes: +``` +DS_STM32F401_datasheet_v1.0.0_2024-08-04.pdf # Auto-moves to docs/datasheets/ +DS_LM358_operational-amplifier_2024-08-04.pdf # Auto-moves to docs/datasheets/ +IMG_pcb-layout_top-view_v1.0.0_2024-08-04.png # Auto-moves to docs/images/ +IMG_assembly-photo_step-3_2024-08-04.jpg # Auto-moves to docs/images/ +IMG_schematic-block-diagram_v2.1.0_2024-08-04.svg # Auto-moves to docs/images/ +``` + +**Prefix Rules:** +- **DS_**: Datasheets - automatically moved to `docs/datasheets/` +- **IMG_**: Images, diagrams, photos - automatically moved to `docs/images/` + #### Manufacturing Files ``` my-project_gerbers_v1.0.0_2024-08-04.zip diff --git a/README.md b/README.md index 593ab42..f7346f6 100644 --- a/README.md +++ b/README.md @@ -160,7 +160,8 @@ This project follows **Semantic Versioning 2.0.0** and **Harvard Data Management - **No spaces**: Use hyphens (-) instead of spaces - **Date format**: YYYY-MM-DD for chronological sorting - **Versioning**: Follow semantic versioning (MAJOR.MINOR.PATCH) -- **Examples**: `my-project_bom_v1.2.0_2024-08-04.csv` +- **CI Auto-Sort**: Use DS_ prefix for datasheets, IMG_ prefix for images +- **Examples**: `my-project_bom_v1.2.0_2024-08-04.csv`, `DS_STM32F401_datasheet_2024-08-11.pdf` ### Initial Setup ```bash diff --git a/ci/README.md b/ci/README.md index d0b010a..f813e66 100644 --- a/ci/README.md +++ b/ci/README.md @@ -1,3 +1,161 @@ # /ci -Continuous Integration configurations and automation scripts. \ No newline at end of file +Continuous Integration configuration and automation scripts. + +## Purpose +This directory contains CI/CD configuration files and automation scripts for the project, including automated file organization, testing, and deployment workflows. + +## CI Workflows + +### Auto-Sort Files (`auto-sort-files.yml`) +Automatically organizes datasheets and images based on filename prefixes. + +#### How It Works +1. **Triggers**: Runs on push to main/develop branches and pull requests +2. **Detection**: Scans the entire repository for files with specific prefixes +3. **Organization**: Moves files to their correct directories +4. **Commit**: Automatically commits the reorganized files + +#### Supported Prefixes +- **DS_**: Datasheets → moved to `docs/datasheets/` +- **IMG_**: Images → moved to `docs/images/` + +#### Example Usage +```bash +# User uploads files anywhere in the project: +/some-folder/DS_STM32F401_datasheet_2024-08-11.pdf +/random-location/IMG_pcb-layout_top-view_v1.0.0_2024-08-11.png + +# CI automatically moves them to: +docs/datasheets/DS_STM32F401_datasheet_2024-08-11.pdf +docs/images/IMG_pcb-layout_top-view_v1.0.0_2024-08-11.png +``` + +#### Supported File Types +**Datasheets (DS_):** +- PDF, DOC, DOCX, TXT + +**Images (IMG_):** +- PNG, JPG, JPEG, SVG, GIF, PDF, WEBP + +#### Configuration +The workflow can be customized by editing `auto-sort-files.yml`: +- Add new file extensions +- Modify destination directories +- Add new prefix categories +- Adjust trigger conditions + +## Benefits of Auto-Sort System + +### For Users +- **No manual organization**: Upload files anywhere with correct prefix +- **Consistent structure**: Files automatically end up in the right place +- **Reduced errors**: No need to remember directory structures +- **Time saving**: Focus on content, not file management + +### For Teams +- **Enforced standards**: Automatic compliance with naming conventions +- **Reduced conflicts**: Files always in predictable locations +- **Better collaboration**: Team members can find files easily +- **Documentation quality**: Consistent organization improves documentation + +### For Maintenance +- **Automated cleanup**: Misplaced files are automatically corrected +- **Audit trail**: Git history shows when files were moved +- **Scalability**: Works regardless of project size +- **Integration**: Works with existing workflows and tools + +## Setup Instructions + +### GitHub Actions +1. The workflow file is already configured in `.github/workflows/` (copy from `ci/auto-sort-files.yml`) +2. Ensure repository has appropriate permissions for the action to commit +3. The workflow will run automatically on pushes and pull requests + +### GitLab CI +To adapt for GitLab CI, create `.gitlab-ci.yml`: +```yaml +auto-sort: + stage: organize + script: + - # Copy the bash script from auto-sort-files.yml + only: + - main + - develop +``` + +### Other CI Systems +The bash script in `auto-sort-files.yml` can be adapted for: +- Jenkins +- Azure DevOps +- CircleCI +- Travis CI +- Any system that can run bash scripts + +## Customization + +### Adding New Prefixes +To add new file type prefixes, modify the workflow: + +```bash +# Example: Add CAD files with CAD_ prefix +find . -name "CAD_*.step" -o -name "CAD_*.sldprt" | while read file; do + if [[ "$file" != ./mechanical/cad/* ]]; then + filename=$(basename "$file") + move_file "$file" "mechanical/cad/$filename" + fi +done +``` + +### Subdirectory Organization +For more sophisticated organization, add logic to sort into subdirectories: + +```bash +# Example: Sort datasheets by component type +case "$filename" in + DS_STM32*|DS_ESP32*) dest_dir="docs/datasheets/microcontrollers/" ;; + DS_LM*|DS_AMS*) dest_dir="docs/datasheets/power-management/" ;; + *) dest_dir="docs/datasheets/" ;; +esac +``` + +## Best Practices + +### File Naming +- Always use the correct prefix (DS_, IMG_) +- Follow the established naming convention +- Include version numbers and dates +- Use descriptive filenames + +### Repository Management +- Review auto-sort commits regularly +- Manually organize files into subdirectories as needed +- Keep the CI workflow updated with project needs +- Monitor for any files that might be missed + +### Team Guidelines +- Train team members on prefix usage +- Document any custom prefixes or rules +- Include prefix requirements in contribution guidelines +- Set up notifications for auto-sort commits + +## Troubleshooting + +### Common Issues +1. **Files not moving**: Check filename prefix and extension +2. **Permission errors**: Ensure CI has write access to repository +3. **Workflow not running**: Check trigger conditions and branch names +4. **Duplicate files**: CI will not overwrite existing files + +### Debugging +- Check workflow logs in CI system +- Verify file extensions are supported +- Ensure files are not already in correct location +- Test bash script locally before deploying + +## Future Enhancements +- Add support for more file types +- Implement smart subdirectory sorting +- Add file validation and quality checks +- Integration with project management tools +- Automated file naming compliance checking diff --git a/ci/auto-sort-files.yml b/ci/auto-sort-files.yml new file mode 100644 index 0000000..5b9487c --- /dev/null +++ b/ci/auto-sort-files.yml @@ -0,0 +1,113 @@ +# CI Auto-Sort Configuration for Datasheets and Images +# This GitHub Actions workflow automatically moves files with specific prefixes to their correct directories + +name: Auto-Sort Files + +on: + push: + branches: [ main, develop ] + pull_request: + branches: [ main ] + +jobs: + auto-sort: + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Auto-sort datasheets and images + run: | + #!/bin/bash + set -e + + # Flag to track if any files were moved + FILES_MOVED=false + + # Function to move files with logging + move_file() { + local src="$1" + local dest="$2" + local dest_dir=$(dirname "$dest") + + # Create destination directory if it doesn't exist + mkdir -p "$dest_dir" + + # Move file if it's not already in the correct location + if [ "$src" != "$dest" ] && [ -f "$src" ]; then + echo "Moving: $src -> $dest" + mv "$src" "$dest" + FILES_MOVED=true + fi + } + + # Find and move datasheet files (DS_ prefix) + echo "Searching for datasheet files with DS_ prefix..." + find . -name "DS_*.pdf" -o -name "DS_*.doc" -o -name "DS_*.docx" -o -name "DS_*.txt" | while read file; do + # Skip files already in docs/datasheets/ + if [[ "$file" != ./docs/datasheets/* ]]; then + filename=$(basename "$file") + move_file "$file" "docs/datasheets/$filename" + fi + done + + # Find and move image files (IMG_ prefix) + echo "Searching for image files with IMG_ prefix..." + find . -name "IMG_*.png" -o -name "IMG_*.jpg" -o -name "IMG_*.jpeg" -o -name "IMG_*.svg" -o -name "IMG_*.gif" -o -name "IMG_*.pdf" -o -name "IMG_*.webp" | while read file; do + # Skip files already in docs/images/ + if [[ "$file" != ./docs/images/* ]]; then + filename=$(basename "$file") + move_file "$file" "docs/images/$filename" + fi + done + + # Check if any files were moved + if [ "$FILES_MOVED" = true ]; then + echo "Files were moved. Setting output for commit step." + echo "files_moved=true" >> $GITHUB_OUTPUT + else + echo "No files needed to be moved." + echo "files_moved=false" >> $GITHUB_OUTPUT + fi + id: sort_files + + - name: Commit moved files + if: steps.sort_files.outputs.files_moved == 'true' + run: | + git config --local user.email "action@github.com" + git config --local user.name "GitHub Action Auto-Sort" + git add docs/datasheets/ docs/images/ + git commit -m "Auto-sort: Move DS_ and IMG_ prefixed files to correct directories + + - Moved datasheet files (DS_*) to docs/datasheets/ + - Moved image files (IMG_*) to docs/images/ + - Automated by CI auto-sort workflow" + + - name: Push changes + if: steps.sort_files.outputs.files_moved == 'true' + uses: ad-m/github-push-action@master + with: + github_token: ${{ secrets.GITHUB_TOKEN }} + branch: ${{ github.ref }} + + - name: Create summary + run: | + echo "## Auto-Sort Results" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + if [ "${{ steps.sort_files.outputs.files_moved }}" = "true" ]; then + echo "✅ Files were automatically moved to correct directories" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "### Moved Files:" >> $GITHUB_STEP_SUMMARY + echo "- Datasheet files (DS_*) → docs/datasheets/" >> $GITHUB_STEP_SUMMARY + echo "- Image files (IMG_*) → docs/images/" >> $GITHUB_STEP_SUMMARY + else + echo "ℹ️ No files needed to be moved - all files are properly organized" >> $GITHUB_STEP_SUMMARY + fi + echo "" >> $GITHUB_STEP_SUMMARY + echo "### Supported Prefixes:" >> $GITHUB_STEP_SUMMARY + echo "- **DS_**: Datasheets (PDF, DOC, DOCX, TXT)" >> $GITHUB_STEP_SUMMARY + echo "- **IMG_**: Images (PNG, JPG, SVG, GIF, PDF, WEBP)" >> $GITHUB_STEP_SUMMARY diff --git a/docs/datasheets/README.md b/docs/datasheets/README.md index 03a12c1..567a97f 100644 --- a/docs/datasheets/README.md +++ b/docs/datasheets/README.md @@ -1,3 +1,90 @@ # /docs/datasheets -Datasheets for key components. \ No newline at end of file +Component datasheets and technical specifications. + +## Purpose +This directory contains datasheets for all components used in the project, providing essential technical information for design, assembly, and troubleshooting. + +## File Organization +Datasheets should be organized by component type or manufacturer for easy reference: + +``` +datasheets/ +├── microcontrollers/ +│ ├── DS_STM32F401_datasheet_v1.0.0_2024-08-11.pdf +│ └── DS_ESP32_datasheet_v2.1.0_2024-08-11.pdf +├── power-management/ +│ ├── DS_AMS1117_voltage-regulator_2024-08-11.pdf +│ └── DS_LM2596_switching-regulator_2024-08-11.pdf +├── sensors/ +│ ├── DS_BME280_environmental-sensor_2024-08-11.pdf +│ └── DS_MPU6050_imu-sensor_2024-08-11.pdf +├── connectors/ +│ ├── DS_USB-C_receptacle_2024-08-11.pdf +│ └── DS_JST-XH_connector-series_2024-08-11.pdf +└── passive-components/ + ├── DS_Yageo_resistor-series_2024-08-11.pdf + └── DS_Murata_capacitor-series_2024-08-11.pdf +``` + +## CI Auto-Sort System +To enable automatic organization by CI bots, use the **DS_** prefix for all datasheet files: + +### Naming Convention +``` +DS_[component-name]_[description]_[version]_[date].pdf +``` + +### Examples +``` +DS_STM32F401_datasheet_v1.0.0_2024-08-11.pdf +DS_LM358_operational-amplifier_2024-08-11.pdf +DS_USB-C_receptacle-specification_2024-08-11.pdf +DS_Yageo_resistor-series-RC_2024-08-11.pdf +``` + +### How It Works +1. **Upload anywhere**: Place datasheet files with DS_ prefix anywhere in the project +2. **CI detection**: CI bot automatically detects files starting with DS_ +3. **Auto-move**: Files are automatically moved to `docs/datasheets/` directory +4. **Organization**: Files can be further organized into subdirectories manually or by additional CI rules + +## File Types +- **PDF**: Primary format for datasheets (preferred) +- **HTML**: Web-based datasheets (save as PDF when possible) +- **DOC/DOCX**: Microsoft Word documents (convert to PDF) +- **TXT**: Plain text specifications (minimal use) + +## Best Practices +- **Always use DS_ prefix** for automatic CI sorting +- **Include version numbers** when available from manufacturer +- **Use descriptive names** that identify the component clearly +- **Download official datasheets** directly from manufacturer websites +- **Keep original filenames** when possible, but add DS_ prefix +- **Include date** when datasheet was downloaded/updated +- **Verify accuracy** - ensure datasheet matches actual component used + +## Datasheet Sources +- **Manufacturer websites**: Primary source for official datasheets +- **Distributor sites**: Digi-Key, Mouser, LCSC often have datasheets +- **Component databases**: Octopart, Findchips for cross-referencing +- **Avoid unofficial sources**: Use only verified manufacturer data + +## Integration with BOM +Datasheets should correspond to components in your BOM: +- Reference the datasheet URL in BOM files +- Ensure part numbers match between BOM and datasheets +- Include datasheet links in component selection documentation +- Use datasheets to verify component specifications during design + +## Version Control +- **Commit datasheets** with design files for complete project history +- **Tag important versions** when component specifications change +- **Document changes** when updating to newer datasheet versions +- **Maintain traceability** between design decisions and datasheet specifications + +## File Size Considerations +- **Compress large files** when possible without losing quality +- **Split large documents** into relevant sections if needed +- **Use Git LFS** for very large files (>100MB) if necessary +- **Consider external links** for extremely large manufacturer documents diff --git a/docs/images/README.md b/docs/images/README.md index eccfdd0..76b5c76 100644 --- a/docs/images/README.md +++ b/docs/images/README.md @@ -1,3 +1,154 @@ # /docs/images -Diagrams, photos, and figures. \ No newline at end of file +Documentation images, diagrams, photos, and figures. + +## Purpose +This directory contains all visual documentation for the project including diagrams, photos, screenshots, and illustrations used in documentation, assembly guides, and technical references. + +## File Organization +Images should be organized by type and purpose for easy reference: + +``` +images/ +├── schematics/ +│ ├── IMG_schematic-block-diagram_v1.0.0_2024-08-11.svg +│ ├── IMG_power-supply-schematic_v1.2.0_2024-08-11.png +│ └── IMG_signal-flow-diagram_v2.0.0_2024-08-11.pdf +├── pcb-layouts/ +│ ├── IMG_pcb-layout_top-view_v1.0.0_2024-08-11.png +│ ├── IMG_pcb-layout_bottom-view_v1.0.0_2024-08-11.png +│ └── IMG_pcb-3d-render_assembled_v1.0.0_2024-08-11.jpg +├── assembly/ +│ ├── IMG_assembly-photo_step-1_2024-08-11.jpg +│ ├── IMG_assembly-photo_step-2_2024-08-11.jpg +│ └── IMG_assembly-diagram_exploded-view_2024-08-11.png +├── mechanical/ +│ ├── IMG_enclosure-design_front-view_v2.0.0_2024-08-11.png +│ ├── IMG_mounting-bracket_dimensions_v1.1.0_2024-08-11.pdf +│ └── IMG_mechanical-assembly_3d-view_v1.0.0_2024-08-11.jpg +├── testing/ +│ ├── IMG_test-setup_functional_2024-08-11.jpg +│ ├── IMG_oscilloscope-capture_signal-analysis_2024-08-11.png +│ └── IMG_measurement-results_performance_2024-08-11.png +└── user-interface/ + ├── IMG_gui-screenshot_main-window_v1.0.0_2024-08-11.png + ├── IMG_display-layout_status-screen_v1.0.0_2024-08-11.jpg + └── IMG_control-panel_button-layout_2024-08-11.png +``` + +## CI Auto-Sort System +To enable automatic organization by CI bots, use the **IMG_** prefix for all image files: + +### Naming Convention +``` +IMG_[subject]_[description]_[version]_[date].extension +``` + +### Examples +``` +IMG_pcb-layout_top-view_v1.0.0_2024-08-11.png +IMG_assembly-photo_step-3_2024-08-11.jpg +IMG_schematic-block-diagram_v2.1.0_2024-08-11.svg +IMG_test-results_frequency-response_2024-08-11.png +IMG_enclosure-design_front-panel_v1.2.0_2024-08-11.pdf +``` + +### How It Works +1. **Upload anywhere**: Place image files with IMG_ prefix anywhere in the project +2. **CI detection**: CI bot automatically detects files starting with IMG_ +3. **Auto-move**: Files are automatically moved to `docs/images/` directory +4. **Organization**: Files can be further organized into subdirectories manually or by additional CI rules + +## Supported File Types +- **PNG**: Preferred for diagrams, screenshots, and technical illustrations +- **JPG/JPEG**: Best for photographs and complex images +- **SVG**: Vector graphics for scalable diagrams and schematics +- **PDF**: Vector documents and technical drawings +- **GIF**: Animated demonstrations (use sparingly) +- **WEBP**: Modern format for web optimization (when supported) + +## Image Categories + +### Schematic Diagrams +- Block diagrams showing system architecture +- Circuit schematics and sub-circuits +- Signal flow and timing diagrams +- Electrical connection diagrams + +### PCB and Layout Images +- PCB layout views (top, bottom, internal layers) +- 3D renderings of assembled boards +- Component placement diagrams +- Routing and trace illustrations + +### Assembly Documentation +- Step-by-step assembly photos +- Component identification images +- Tool and equipment photos +- Before/after assembly comparisons + +### Mechanical Designs +- CAD renderings and technical drawings +- Enclosure and housing designs +- Mounting and installation diagrams +- Dimensional drawings and specifications + +### Testing and Validation +- Test setup photographs +- Oscilloscope captures and measurements +- Performance graphs and charts +- Compliance testing results + +### User Interface +- GUI screenshots and mockups +- Display layouts and screen designs +- Control panel and button layouts +- User interaction flows + +## Best Practices +- **Always use IMG_ prefix** for automatic CI sorting +- **Use descriptive names** that clearly identify the image content +- **Include version numbers** for design-related images +- **Optimize file sizes** without compromising quality +- **Use appropriate formats** (PNG for diagrams, JPG for photos) +- **Include date** when image was created or captured +- **Maintain consistent naming** across related images +- **Document image sources** and creation methods + +## Image Quality Guidelines +- **Resolution**: Use appropriate resolution for intended use + - Screenshots: Native resolution or 1920x1080 max + - Photos: 1920x1080 to 4K depending on detail needed + - Diagrams: Vector format (SVG) when possible +- **Compression**: Balance file size with quality +- **Color space**: Use sRGB for web compatibility +- **File size**: Keep under 5MB per image when possible + +## Creation Tools +- **Screenshots**: Built-in OS tools, Greenshot, LightShot +- **Diagrams**: Draw.io, Lucidchart, Visio, Inkscape +- **Photo editing**: GIMP, Photoshop, Paint.NET +- **CAD renders**: KiCad 3D viewer, FreeCAD, SolidWorks +- **Vector graphics**: Inkscape, Illustrator, SVG editors + +## Integration with Documentation +Images should be referenced in documentation: +- Use relative paths in markdown: `![Description](../images/IMG_filename.png)` +- Include alt text for accessibility +- Reference images by filename in assembly guides +- Maintain links when reorganizing files +- Update documentation when images are updated + +## Version Control +- **Commit images** with related design changes +- **Use Git LFS** for large image files (>10MB) +- **Tag versions** when images represent specific design iterations +- **Document changes** when updating existing images +- **Maintain history** of design evolution through images + +## File Size Management +- **Optimize images** before committing to reduce repository size +- **Use appropriate compression** for different image types +- **Consider external hosting** for very large images +- **Regular cleanup** of outdated or unused images +- **Monitor repository size** and use Git LFS when necessary