Skip to content

Conversation

@fpjentzsch
Copy link

@fpjentzsch fpjentzsch commented Jan 29, 2025

Adds a regression testing and benchmarking suite to the GitLab CI, next to the existing unit test test suite. Includes:

  • Benchmarking scripts that wrap the builder to perform gridsearches defined in a simple YAML format
  • Infrastructure for running bitstreams on an FPGA board and logging all results via DVC
  • New custom_step_library to collect all custom steps required for the included end2end models (stored via DVC)
  • Extension of the validate.py script for on-board Top-1 accuracy validation on MNIST, CIFAR-10, CIFAR-100, ImageNet (if synthesized with IODMA shell)
  • Changes to the logging infrastructure and launch_process_helper

See #76 for more information.

Depends on / includes:

@fpjentzsch fpjentzsch self-assigned this Jan 29, 2025
@fpjentzsch fpjentzsch changed the title CI for Benchmarking CI for Regression Testing & Benchmarking May 23, 2025
@fpjentzsch fpjentzsch marked this pull request as ready for review May 23, 2025 11:47
@fpjentzsch fpjentzsch merged commit e4efa8a into dev May 28, 2025
1 of 2 checks passed
@fpjentzsch fpjentzsch moved this from In Progress to Merged into FINN+ in FINN+ Feature Tracker May 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Merged into FINN+

Development

Successfully merging this pull request may close these issues.

1 participant