-
Notifications
You must be signed in to change notification settings - Fork 83
Open
Description
Feature Description
We now track model runtimes when run by the CI
If we start reporting them in a public-facing website we should include some disclaimers because the CI isnt exactly running these models optimally. The difference between a run where you are going for performance and a CI run could be very high, probably at least 5x imo.
We don't want the VTB to be helping external actors draw bad conclusions about the performance of the codes
Design
Disclaimers
A help page on how to run the codes optimally ?
Impact
Stick to the original mission of the VTB
Metadata
Metadata
Assignees
Labels
No labels