Skip to content

DFBUGS-1628: increase threshold for OSDCPULoadHigh #3236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

DFBUGS-1628: increase threshold for OSDCPULoadHigh #3236

wants to merge 1 commit into from

Conversation

weirdwiz
Copy link
Contributor

@weirdwiz weirdwiz commented May 16, 2025 •
edited
Loading

this commit increases the threshold for the alert to avoid false positives in performance mode. the thresholds have been set after discussion on the BZ

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented May 16, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: weirdwiz
Once this PR has been reviewed and has the lgtm label, please assign travisn for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@weirdwiz weirdwiz changed the title increase threshold for OSDCPULoadHigh DFBUGS 1628: increase threshold for OSDCPULoadHigh May 16, 2025
@weirdwiz weirdwiz changed the title DFBUGS 1628: increase threshold for OSDCPULoadHigh DFBUGS-1628: increase threshold for OSDCPULoadHigh May 16, 2025
@openshift-ci-robot openshift-ci-robot added jira/severity-important jira/valid-reference Indicates that this PR references a valid jira ticket of any type jira/invalid-bug Indicates that the referenced jira bug is invalid for the branch this PR is targeting labels May 16, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented May 16, 2025 •
edited by openshift-ci bot
Loading

@weirdwiz: This pull request references [Jira Issue DFBUGS-1628](https://issues.redhat.com//browse/DFBUGS-1628), which is invalid:

  • expected the bug to target the "odf-4.18" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

this commit increases the threshold for the alert to avoid false positives in performance mode.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@weirdwiz
Copy link
Contributor Author

can you PTAL @umangachapagain

@openshift-ci-robot
Copy link

openshift-ci-robot commented May 19, 2025 •
edited by openshift-ci bot
Loading

@weirdwiz: This pull request references [Jira Issue DFBUGS-1628](https://issues.redhat.com//browse/DFBUGS-1628), which is invalid:

  • expected the bug to target either version "odf-4.18." or "openshift-odf-4.18.", but it targets "odf-4.19" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

this commit increases the threshold for the alert to avoid false positives in performance mode. the thresholds have been set after discussion on the BZ

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@weirdwiz
increase threshold for OSDCPULoadHigh
93db1d6 
this commit increases the threshold for the alert to avoid false
positives.

Signed-off-by: Divyansh Kamboj <[email protected]>
umangachapagain
umangachapagain requested changes May 20, 2025
View reviewed changes
metrics/deploy/prometheus-ocs-rules.yaml
Comment on lines +378 to +381
pod:container_cpu_usage:sum{pod=~"rook-ceph-osd-.*"} / on(pod) kube_pod_resource_limit{resource='cpu',pod=~"rook-ceph-osd-.*"} > 0.90
for: 12h
labels:
severity: warning
severity: info
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of changes required:

  1. Update the description and message to reflect the new threshold and "12h" period.
  2. Update the runbook as well.
  3. Update severity_level annotation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@umangachapagain umangachapagain umangachapagain requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@umangachapagain umangachapagain

Labels
jira/invalid-bug Indicates that the referenced jira bug is invalid for the branch this PR is targeting jira/severity-important jira/valid-reference Indicates that this PR references a valid jira ticket of any type
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@weirdwiz @openshift-ci-robot @umangachapagain