Description
We have some long running behat jobs in CI, most notably elemental which takes over 45 minutes. This slowness means that developers working on pull-requests need to wait a long time for CI to complete, which isn't great.
We should be able to split behat jobs into multi jobs by using @tags on the behat feature files to denote which jobs they should belong to e.g. @job1 and then updating gha-generate-matrix to dynmically detect these tags are create a number of jobs and then on the command line add a --tags @job1 command line option when running behat.
We should look at existing behat jobs and split things if they're over 10 minutes running time. I've used 10 minutes as that's roughly how long the framework-orm phpunit job takes, and that's a large phpunit job.
Acceptance criteria
- Long running behat CI job are identified
- Behat CI jobs are split if they're over 10 minutes in running time
- An exception is thrown from gha-generate-matrix if a feature file with a
@job1style tag is found, and there is another feature file that does not have a job tag. This is done because otherwise this feature file will never be run in CI - Job tag are automatically detected by the CI. Behat keeps running normally if there's no job tag detected.
Example run:
https://github.com/emteknetnz/silverstripe-admin/actions/runs/9947426641
(note: this test had jobs split 2x, for the actual PR I've split them 3x)
PRs
- NEW Add endtoend_tag param gha-run-tests#31
- NEW Output endtoend_tag gha-generate-matrix#99
- ENH Pass in endtoend_tag gha-ci#136
- MNT Split behat jobs silverstripe-admin#1794
- MNT Split behat jobs silverstripe-asset-admin#1476
- MNT Split behat jobs silverstripe-cms#2975
- MNT Split behat jobs silverstripe-linkfield#307
- MNT Split behat jobs silverstripe-versioned-admin#353
- FIX Validate correct input gha-run-tests#33
- FIX Truncate table to clear table silverstripe-framework#11309
- MNT Split behat jobs silverstripe-elemental#1219
- MNT Split behat jobs recipe-kitchen-sink#67