refactor: Modernize Docker setup and improve connector scripts

[nikagra]

Modernize cp-quickstart Docker Compose and Scripts

This PR updates the cp-quickstart environment for the Scylla CDC Source Connector to improve compatibility, reliability, and user experience with modern Docker Compose and Confluent images.

Key changes

• docker-compose.yml
  • Use official Confluent Kafka Connect image directly (removes custom Dockerfile).
  • Mount connector JARs to /usr/share/confluent-hub-components and set CONNECT_PLUGIN_PATH accordingly.
  • Set SCYLLA_CLUSTER_NAME to scylla_cluster (underscore, not dash) for consistency with topic naming and documentation (fixes Avro warning).
• setup-containers.sh
  • Auto-detects and uses either docker compose (plugin) or legacy docker-compose.
  • Waits for Kafka Connect REST API to be ready before proceeding.
  • Prints loaded Scylla JARs and plugin discovery logs for easier troubleshooting.
• setup-connector.sh
  • Waits for Kafka Connect REST API and plugin scan before registering connector.
  • Verifies ScyllaConnector plugin is available before attempting registration.
  • Uses correct topic prefix and cluster name in all output and configuration.
  • Updates consumer example to use scylla_cluster.demo_keyspace.users.
• cleanup.sh
  • Uses the same compose command detection logic for stopping/removing containers.
• README.md
  • Updates all topic and cluster name references to scylla_cluster (underscore).
  • Updates consumer example and JSON output to match new topic naming.

Motivation

• Ensures compatibility with both docker compose and docker-compose.
• Avoids confusion with cluster/topic naming (dash vs underscore).
• Simplifies plugin deployment and troubleshooting for new users.
• Reduces friction for local development and onboarding.

Testing

• Confirmed connector JARs are loaded and plugin is detected.
• Confirmed CDC events are published to the correct topic.

---

[Copilot]

Pull request overview

This PR modernizes the cp-quickstart Docker environment by replacing a custom Dockerfile with the official Confluent Kafka Connect image and improving setup scripts with better compatibility and error handling.

Key changes:

• Replaced custom Dockerfile with official confluentinc/cp-kafka-connect:8.1.1 image using volume mounts for connector JARs
• Added auto-detection for both docker compose (plugin) and legacy docker-compose commands
• Changed Scylla cluster name from scylla-cluster to scylla_cluster for consistency with topic naming conventions

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
cp-quickstart/docker-compose.yml	Switched to official Confluent image, updated plugin path to use volume mount, and changed cluster name to underscore format
cp-quickstart/setup-containers.sh	Added docker compose command detection, REST API readiness checks, and JAR verification logging
cp-quickstart/setup-connector.sh	Enhanced with plugin verification, connector update/create logic, and corrected topic naming in examples
cp-quickstart/cleanup.sh	Added docker compose command detection for consistency with setup script
cp-quickstart/README.md	Updated documentation to reflect new cluster name and topic naming conventions
cp-quickstart/Dockerfile.connect	Removed custom Dockerfile as it's replaced by official image with volume mounts

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.