Skip to content

size based load balancing: add test #10992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

cezarmoise
Copy link
Contributor

@cezarmoise cezarmoise commented May 29, 2025

Warning

DRAFT

Test to ensure that the cluster is balanced correctly in difficult conditions:

  • heterogeneous nodes with different disk sizes.
  • multiple tables with different partition sizes.

This test will:

  1. Expand the cluster by adding new nodes.
  2. Run the original test_custom_time to populate the cluster with data.
  3. Wait for tablet migration to finish.
  4. Check the disk usage of each node to ensure they are balanced.

Cluster setup: 1 DC, 3 racks, 3 nodes per rack (1x i4i.large, 1x i4i.xlarge, 1x i4i.2xlarge)
The test creates 3 tables:

  • table 1
    • 200000 x 1MB partitions
    • 10000 x 10MB partitions
    • 200 x 1GB partitions
    • 10 x 10GB partitions
  • table 2
    • 100000 x 1MB partitions
    • 2000 x 100MB partitions
    • 100 x 1GB partitions
    • 2 x 100GB partitions
  • table 3
    • 20000 x 10MB partitions
    • 1000 x 100MB partitions
    • 20 x 10GB partitions
    • 1 x 100GB partitions

image

Testing

PR pre-checks (self review)

  • I added the relevant backport labels
  • I didn't leave commented-out/debugging code

Reminders

  • Add New configuration option and document them (in sdcm/sct_config.py)
  • Add unit tests to cover my changes (under unit-test/ folder)
  • Update the Readme/doc folder relevant to this change (if needed)

@cezarmoise cezarmoise requested a review from nodep May 29, 2025 14:16
@cezarmoise cezarmoise self-assigned this May 29, 2025
@cezarmoise cezarmoise added the backport/none Backport is not required label May 29, 2025
@scylladbbot
Copy link

@cezarmoise new branch manager-3.6 was added, please add backport label if needed

@cezarmoise cezarmoise force-pushed the feature/size-based-load-balancing branch from ff69e7a to ddbfe8b Compare June 30, 2025 08:05
@scylladbbot
Copy link

@cezarmoise new branch branch-2025.3 was added, please add backport label if needed

@cezarmoise cezarmoise force-pushed the feature/size-based-load-balancing branch from ddbfe8b to 93a2673 Compare July 1, 2025 09:11
check node storage is balanced after writes finish
@cezarmoise cezarmoise force-pushed the feature/size-based-load-balancing branch 2 times, most recently from 5d7d20d to 50786e7 Compare July 10, 2025 19:04
@cezarmoise cezarmoise force-pushed the feature/size-based-load-balancing branch from 50786e7 to 56485f0 Compare July 10, 2025 19:05
@cezarmoise cezarmoise force-pushed the feature/size-based-load-balancing branch from 65b29e4 to c783b37 Compare July 11, 2025 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/none Backport is not required
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants