🌱 Adds overlays for release v0.12

[Sunnatillo]

What this PR does / why we need it:
Adds overlays for release v0.12

Fixes #

Checklist:

• Documentation has been updated, if necessary.
• Unit tests have been added, if necessary.
• E2E tests have been added, if necessary.
• Integration tests have been added, if necessary.

[Sunnatillo]

/test ?

[metal3-io-bot]

@Sunnatillo: The following commands are available to trigger required jobs:

/test generate

/test gomod

/test manifestlint

/test markdownlint

/test shellcheck

/test test

The following commands are available to trigger optional jobs:

/test metal3-bmo-e2e-test-optional-pull

/test metal3-centos-e2e-basic-test-main

/test metal3-centos-e2e-feature-test-main-features

/test metal3-centos-e2e-feature-test-main-pivoting

/test metal3-centos-e2e-feature-test-main-remediation

/test metal3-centos-e2e-integration-k8s-pre-release-test-main

/test metal3-centos-e2e-integration-test-main

/test metal3-dev-env-integration-test-centos-main

/test metal3-dev-env-integration-test-ubuntu-main

/test metal3-e2e-1-33-1-34-upgrade-test-main

/test metal3-e2e-clusterctl-upgrade-test-main

/test metal3-ubuntu-e2e-basic-test-main

/test metal3-ubuntu-e2e-feature-test-main-features

/test metal3-ubuntu-e2e-feature-test-main-pivoting

/test metal3-ubuntu-e2e-feature-test-main-remediation

/test metal3-ubuntu-e2e-integration-k8s-pre-release-test-main

/test metal3-ubuntu-e2e-integration-test-main

Use /test all to run the following jobs that were automatically triggered:

generate

gomod

manifestlint

shellcheck

test

Details

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[Sunnatillo]

/test metal3-bmo-e2e-test-optional-pull

[Sunnatillo]

/cc @lentzi90 @tuminoid @Rozzii

[adilGhaffarDev]

/lgtm

[metal3-io-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: adilGhaffarDev
Once this PR has been reviewed and has the lgtm label, please assign zaneb for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kashifest]

@lentzi90 I wrongly commented on the other PR, how about this one? If this is adding the overlays, what is running on the PRs then?

[lentzi90]

This is adding config for the optional tests (i.e. upgrade). These tests doesn't make much sense to trigger on release branches. It would run the upgrade tests that existed when the release was cut. We run them from main to check upgrades from all supported versions

[lentzi90]

I think the tests fail because we run an older version of IrSO:

baremetal-operator/test/e2e/data/ironic-standalone-operator/operator/kustomization.yaml

Line 6 in 2d32335

- https://github.com/metal3-io/ironic-standalone-operator/releases/download/v0.5.2/install.yaml

. It does not know about Ironic v32 and v33.
We will need to coordinate so that we test versions of IrSO and Ironic that work together. Maybe we can remove some of the older Ironic tests and Bump IrSO? Or in worst case we will need to use multiple IrSO versions in these tests, maybe also upgrading IrSO.

[kashifest]

This is adding config for the optional tests (i.e. upgrade). These tests doesn't make much sense to trigger on release branches. It would run the upgrade tests that existed when the release was cut. We run them from main to check upgrades from all supported versions

Ok thanks for explaining this, I understood as this was adding overlays for release branch, the PR e2e tests should use this on PRs for release-0.12

[metal3-io-bot]

New changes are detected. LGTM label has been removed.

[Sunnatillo]

/test metal3-bmo-e2e-test-optional-pull

[lentzi90]

This will not work for the older versions. IrSO v0.7.0 only supports Ironic v31 - v33. I still see tests with Ironic v28 - v30 in the config. These will break.

[Sunnatillo]

Should we parameterize this?

[lentzi90]

No we should not parameterize. If we must test more Ironic versions than the latest IrSO supports, we will need multiple IrSO versions. Just make a copy with the newer version, like we have for each version of Ironic. We may need to add the IrSO overlay also to the e2e upgrade config so it can be different per test.

For example we could test this:

• IrSO v0.5 | Ironic v28 -> v31 ("latest" for IrSO v0.5)
• IrSO v0.5 | Ironic v29 -> v31
• IrSO v0.5 | Ironic v30 -> v31
• IrSO v0.7 | Ironic v31 -> latest (v33 for IrSO v0.7)
• IrSO v0.7 | Ironic v32 -> latest
• IrSO v0.7 | Ironic v33 -> latest <-- This we can remove since there is no newer version than v33.

Note that IrSO v0.5 cannot use Ironic > v31. When we upgrade to "latest" we really upgrade to the latest supported version by IrSO.

[metal3-io-bot]

@Sunnatillo: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
metal3-bmo-e2e-test-optional-pull	7dcea7d	link	false	/test metal3-bmo-e2e-test-optional-pull

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[metal3-io-bot]

@Sunnatillo: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
metal3-bmo-e2e-test-optional-pull	7dcea7d	link	false	/test metal3-bmo-e2e-test-optional-pull

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.