Skip to content

roachtest: deflake splits/load/ycsb/e #149535

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 4, 2025
Merged

Conversation

iskettaneh
Copy link
Contributor

@iskettaneh iskettaneh commented Jul 4, 2025

This test has a long history of flakes. The main reason is that at the first couple of minutes of the test, there many splits could happen because of the init phase of the workload increases the CPU and it has many inserts. However, for the remainder of the test, the workload is mostly scans, and that shouldn't cause many splits.

This commit increases the maximumRanges allowed by 2. We ~recently increased it to
the current number and saw almost now flakes, so increasing it by 2 should hopefully
be enough.

In the future, if we see many flakes of this nature. I think we should ignore the splits that happen in the first 2 minutes of the test, and focus on the fact that the splits in the remainder of the test are minimal.

Fixes: #149516

Release Note: None

This test has a long history of flakes. The main reason is that at the first couple of
minutes of the test, there many splits could happen because of the init phase of the
workload increases the CPU and it has many inserts. However, for the remainder of the
test, the workload is mostly scans, and that shouldn't cause many splits.

This commit increases the maximumRanges allowed by 2. We ~recently increased it to
 the current number and saw almost now flakes, so increasing it by 2 should hopefully
 be enough.

In the future, if we see many flakes of this nature. I think we should ignore the splits
that happen in the first 2 minutes of the test, and focus on the fact that the splits in
the remained of the test are minimal.

Fixes: cockroachdb#149516

Release Note: None
@iskettaneh iskettaneh requested review from arulajmani and wenyihu6 July 4, 2025 14:24
@iskettaneh iskettaneh added the backport-all Flags PRs that need to be backported to all supported release branches label Jul 4, 2025
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator

@arulajmani arulajmani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @wenyihu6)

@iskettaneh
Copy link
Contributor Author

TFTR!

bors r+

@craig
Copy link
Contributor

craig bot commented Jul 4, 2025

Copy link

blathers-crl bot commented Jul 4, 2025

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

error creating merge commit from 5f370be to blathers/backport-release-23.2-149535: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch release-23.2 failed. See errors above.


💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

error creating merge commit from 5f370be to blathers/backport-release-24.1-149535: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch release-24.1 failed. See errors above.


💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-all Flags PRs that need to be backported to all supported release branches backport-failed target-release-25.4.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

roachtest: splits/load/ycsb/e/nodes=3/obj=cpu failed
3 participants