Apache Flink CDC Examples

This repository contains comprehensive examples of Apache Flink Change Data Capture (CDC) implementations using different approaches.

Projects

Flink_CDC_Java

Java-based Flink CDC implementation with SQL Server and PostgreSQL integration.

Features:

• Custom Java CDC jobs
• SQL Server → PostgreSQL pipeline
• Maven-based build system
• JAR file deployment

Architecture: SQL Server → Flink (Java) → PostgreSQL

📖 View Documentation | 📖 Türkçe Dokümantasyon

---

Flink_CDC_Yaml

YAML-based Flink CDC configuration with SQL Server and Kafka integration.

Features:

• YAML pipeline configuration
• SQL Server → Kafka streaming
• No-code CDC setup
• Kafka UI monitoring

Architecture: SQL Server → Flink CDC (YAML) → Kafka

📖 View Documentation | 📖 Türkçe Dokümantasyon

Quick Start

Choose your preferred approach:

1. Java Development: Use Flink_CDC_Java for custom CDC logic and PostgreSQL integration
2. Configuration-Based: Use Flink_CDC_Yaml for quick setup with Kafka streaming

Requirements

• Docker & Docker Compose
• 4GB+ RAM
• Available ports: 8081, 1433, 5432/9092, 8080, 8978

Getting Started

# Clone the repository
git clone https://github.com/AhmetFurkanDEMIR/Flink-CDC.git
cd Flink-CDC

# Choose your approach
cd Flink_CDC_Java    # For Java-based implementation
# OR
cd Flink_CDC_Yaml    # For YAML-based implementation

# Start the environment
docker compose up

Services

Service	Port	Description
Flink Web UI	8081	Job management and monitoring
SQL Server	1433	Source database
PostgreSQL	5432	Target database (Java version)
Kafka	9092	Message broker (YAML version)
Kafka UI	8080	Kafka management (YAML version)
CloudBeaver	8978	Database administration