feat(demo): Auto Balancing on Scale-Up

[daniel-y]

Task

Create a demo to showcase automatic rebalancing when scaling up the cluster.

Background

When a new broker is added to a cluster (scale-up), it should immediately start taking on some of the existing load to improve overall capacity. This demo will show how AutoMQ's auto-balancer automatically migrates partitions to a newly added broker.

Proposed Solution / Demo Plan

1. Environment Setup:

   • Create a docker-compose.yml that initially starts with a 2-node AutoMQ cluster.
   • The compose file should have a third broker service defined but not started.

2. Initial State:

   • Start the 2-node cluster and produce a balanced load to a topic with several partitions.
   • Instruct the user to verify that partitions are distributed across the two running brokers.

3. Scale-Up Event:

   • Instruct the user to scale up the cluster by starting the third broker container using the command: docker compose up --scale automq-broker=3 -d.

4. Observation and Verification:

   • Allow the auto-balancer a few minutes to run.
   • Instruct the user to re-check the partition distribution.
   • They should observe that some partitions have been automatically moved from the original two brokers to the newly added third broker, resulting in a balanced cluster.

Acceptance Criteria

• A runnable docker-compose.yml is provided in auto-balancing-demos/scale-up/.
• The README.md provides a clear, step-by-step guide.
• The guide explains how to start a 2-node cluster, scale it up to 3 nodes, and verify that partitions are automatically migrated to the new node.