-
Notifications
You must be signed in to change notification settings - Fork 2.8k
sig-testing: add alpha/beta enabled and conformance periodic jobs #35621
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This mirrors the presubmit counterparts that were added in kubernetes#35534 and tested in kubernetes/kubernetes#134250. I can help monitor the existing alpha/beta features jobs. Those need to be renamed in testgrid to avoid confusion. Other than that they remain unchanged for now. Potential future work: - Changes related to serial and/or slow jobs. Serial tests are excluded implicitly by e2e-k8s.sh because the jobs enables PARALLEL (kubernetes#35594). Slow jobs are disabled in LABEL_FILTER because that is what the existing periodics did. We might be able to run them because as long as they overlap with other tests there shouldn't be much impact on overall job duration (same applies to presubmits!). Scheduling of slow tests may be relevant (onsi/ginkgo#1599). - Release informing/blocking. The existing jobs are release informing. alpha-beta-features shouldn't be because breaking alpha tests is not something that the release team should have to deal with. Instead, the new jobs should get promoted once they are known to be stable. beta-features can remain release informing, tests for beta features (even if off-by-default) need to be stable. - Decision about "enabled-conformance". The conformance jobs got included because it was suggested on Slack. They run a subset of the tests run by their "enabled" counterparts. It remains to be seen whether having two jobs instead of one really provides a better release signal.
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: aojea, pohly The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@pohly: Updated the
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
This mirrors the presubmit counterparts that were added in #35534 and tested in kubernetes/kubernetes#134250.
I can help monitor the existing alpha/beta features jobs. Those need to be renamed in testgrid to avoid confusion. Other than that they remain unchanged for now.
Potential future work:
Changes related to serial and/or slow jobs.
Serial tests are excluded implicitly by e2e-k8s.sh because the jobs enables PARALLEL (kind jobs: stop relying on implicit SKIP=[Serial] #35594).
Slow jobs are disabled in LABEL_FILTER because that is what the existing periodics did. We might be able to run them because as long as they overlap with other tests there shouldn't be much impact on overall job duration (same applies to presubmits!). Scheduling of slow tests may be relevant (prioritize tests during parallel execution onsi/ginkgo#1599).
Release informing/blocking.
The existing jobs are release informing. alpha-beta-features shouldn't be because breaking alpha tests is not something that the release team should have to deal with. Instead, the new jobs should get promoted once they are known to be stable. beta-features can remain release informing, tests for beta features (even if off-by-default) need to be stable.
Decision about "enabled-conformance".
The conformance jobs got included because it was suggested on Slack. They run a subset of the tests run by their "enabled" counterparts. It remains to be seen whether having two jobs instead of one really provides a better release signal.
/assign @aojea @BenTheElder