Skip to content

CI for Regression Testing & Benchmarking #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 154 commits into from
May 28, 2025
Merged

CI for Regression Testing & Benchmarking #34

merged 154 commits into from
May 28, 2025

Conversation

fpjentzsch
Copy link

@fpjentzsch fpjentzsch commented Jan 29, 2025

Adds a regression testing and benchmarking suite to the GitLab CI, next to the existing unit test test suite. Includes:

  • Benchmarking scripts that wrap the builder to perform gridsearches defined in a simple YAML format
  • Infrastructure for running bitstreams on an FPGA board and logging all results via DVC
  • New custom_step_library to collect all custom steps required for the included end2end models (stored via DVC)
  • Extension of the validate.py script for on-board Top-1 accuracy validation on MNIST, CIFAR-10, CIFAR-100, ImageNet (if synthesized with IODMA shell)
  • Changes to the logging infrastructure and launch_process_helper

See #76 for more information.

Depends on / includes:

@fpjentzsch fpjentzsch self-assigned this Jan 29, 2025
@fpjentzsch fpjentzsch changed the title CI for Benchmarking CI for Regression Testing & Benchmarking May 23, 2025
@fpjentzsch fpjentzsch marked this pull request as ready for review May 23, 2025 11:47
@fpjentzsch fpjentzsch merged commit e4efa8a into dev May 28, 2025
1 of 2 checks passed
@fpjentzsch fpjentzsch moved this from In Progress to Merged into FINN+ in FINN+ Feature Tracker May 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Merged into FINN+
Development

Successfully merging this pull request may close these issues.

1 participant