Skip to content

Cassandra scale-out workflow should wait for last replica being scaled-out #182

@pchmieli

Description

@pchmieli

Bug Report

What did you do?

  • Created 3 nodes cluster (3 racks, one node per rack).
  • Generated some data with cassandra-stress cassandra-stress write n=1500000 -rate threads=100 -col "n=FIXED(1) size=FIXED(1000)
  • Scaled-out to 6 nodes (nodesPerRacks=2)

What did you expect to see?
Custom resource in phase Running when no nodes are joining the cluster

What did you see instead? Under which circumstances?
Custom resource in phase Running when last node is still joining (nodetool status still shows UJ, while pod is Running and main container ready)

Environment

  • casskop version:
    2.3.0

  • Kubernetes version information:
    Azure: v1.32.6
    GCP: v1.32.9-gke.1072000

  • Kubernetes cluster kind:
    Azure / GCP

  • Cassandra version:
    4.1.10

Possible Solution
Casskop waits for hasJoiningNodes=false before scaling-out the statefulSet.
Then, it marks ScaleOut=Done for rack just after new pod is Running and main container ready.
Instead, it should wait for hasJoiningNodes=false also befor marking rack as ScaleOut=Done

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions