Skip to content

Conversation

dgoodwin
Copy link
Contributor

@dgoodwin dgoodwin commented Sep 4, 2025

No description provided.

@dgoodwin dgoodwin changed the title Add flaky kubectl logs test to the retry list to clean signal for release OCPBUGS-61287: Add flaky kubectl logs test to the retry list to clean signal for release Sep 4, 2025
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Sep 4, 2025
@openshift-ci-robot
Copy link

@dgoodwin: This pull request references Jira Issue OCPBUGS-61287, which is invalid:

  • expected the bug to target the "4.21.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@dgoodwin
Copy link
Contributor Author

dgoodwin commented Sep 4, 2025

/jira refresh
/verified bypass

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Sep 4, 2025
@openshift-ci-robot
Copy link

@dgoodwin: This pull request references Jira Issue OCPBUGS-61287, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.21.0) matches configured target version for branch (4.21.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

In response to this:

/jira refresh
/verified bypass

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from deads2k and p0lyn0mial September 4, 2025 18:15
@smg247
Copy link
Member

smg247 commented Sep 4, 2025

/lgtm

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 4, 2025
@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 4, 2025
Copy link
Contributor

openshift-ci bot commented Sep 4, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dgoodwin, smg247

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@dgoodwin
Copy link
Contributor Author

dgoodwin commented Sep 5, 2025

/verified bypass

@openshift-ci-robot
Copy link

@dgoodwin: The verified label has been added.

In response to this:

/verified bypass

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Sep 5, 2025
@dgoodwin
Copy link
Contributor Author

dgoodwin commented Sep 5, 2025

/cherry-pick release-4.20

@openshift-cherrypick-robot

@dgoodwin: once the present PR merges, I will cherry-pick it on top of release-4.20 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.20

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD b2728f8 and 2 for PR HEAD cf8fa0f in total

@ardaguclu
Copy link
Member

This PR looks good to me.

I'd like to note that this test failed 4 times on bare metal. In all of them, cluster was experiencing disruption. I think, this test does not have flaky behavior.

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 2af38a7 and 2 for PR HEAD cf8fa0f in total

@dgoodwin
Copy link
Contributor Author

dgoodwin commented Sep 5, 2025

@ardaguclu 4 failures in one week, if you expand the window the pattern is there, it happens time to time.

First run I checked did not correlate with disruption, no disruption observed at all during this run: https://sippy.dptools.openshift.org/sippy-ng/job_runs/1961222114115588096/periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-techpreview/intervals?end=2025-08-29T02%3A52%3A38Z&filterText=&intervalFile=e2e-timelines_spyglass_20250829-014758.json&overrideDisplayFlag=0&selectedSources=OperatorAvailable&selectedSources=OperatorProgressing&selectedSources=OperatorDegraded&selectedSources=KubeletLog&selectedSources=EtcdLog&selectedSources=EtcdLeadership&selectedSources=Alert&selectedSources=Disruption&selectedSources=E2EFailed&selectedSources=APIServerGracefulShutdown&selectedSources=KubeEvent&selectedSources=NodeState&selectedSources=CPUMonitor&start=2025-08-29T00%3A34%3A39Z

Another example: https://sippy.dptools.openshift.org/sippy-ng/job_runs/1961919411627495424/periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-ipv6-runc-techpreview/intervals?end=2025-08-31T01%3A20%3A15Z&filterText=&intervalFile=e2e-timelines_spyglass_20250831-001902.json&overrideDisplayFlag=0&selectedSources=OperatorAvailable&selectedSources=OperatorProgressing&selectedSources=OperatorDegraded&selectedSources=KubeletLog&selectedSources=EtcdLog&selectedSources=EtcdLeadership&selectedSources=Alert&selectedSources=Disruption&selectedSources=E2EFailed&selectedSources=APIServerGracefulShutdown&selectedSources=KubeEvent&selectedSources=NodeState&selectedSources=CPUMonitor&start=2025-08-30T22%3A54%3A25Z also no disruption in this run.

I'm not sure what it is exactly, but I don't think the api is disrupted at the time this is happening.

@ardaguclu
Copy link
Member

Thank you for the analysis. Now it makes sense to me. I'll try to spend time on this next week.

Copy link
Contributor

openshift-ci bot commented Sep 6, 2025

@dgoodwin: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-gcp-ovn-techpreview cf8fa0f link false /test e2e-gcp-ovn-techpreview
ci/prow/e2e-aws-ovn-single-node-serial cf8fa0f link false /test e2e-aws-ovn-single-node-serial
ci/prow/e2e-aws-ovn-single-node-upgrade cf8fa0f link false /test e2e-aws-ovn-single-node-upgrade
ci/prow/e2e-azure cf8fa0f link false /test e2e-azure
ci/prow/e2e-aws-ovn-single-node cf8fa0f link false /test e2e-aws-ovn-single-node
ci/prow/e2e-openstack-ovn cf8fa0f link false /test e2e-openstack-ovn
ci/prow/e2e-aws-disruptive cf8fa0f link false /test e2e-aws-disruptive
ci/prow/e2e-aws-ovn-upgrade cf8fa0f link false /test e2e-aws-ovn-upgrade
ci/prow/e2e-aws-ovn cf8fa0f link false /test e2e-aws-ovn
ci/prow/okd-scos-e2e-aws-ovn cf8fa0f link false /test okd-scos-e2e-aws-ovn
ci/prow/e2e-gcp-csi cf8fa0f link false /test e2e-gcp-csi
ci/prow/e2e-metal-ipi-ovn-ipv6 cf8fa0f link unknown /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants