Skip to content

MQTT Source: tasks.max > 1 not working for a single KCQL with shared subscription #1720

@ehsmsh

Description

@ehsmsh

The MQTT source connector only creates multiple tasks, if number of KCQL statements > 1. While a use case is that we have one statement that subscribes to a shared topic and along with connect.mqtt.share.replicate set to true, it can be replicated on multiple tasks and run in parallel for load balancing.
Right now this config would result in only on task running with a single subscription, instead of shared subscription replicated in two tasks:

"connect.mqtt.kcql": "INSERT INTO telemetry_vehicles_gps SELECT * FROM $share/connect/telemetry/vehicles/gps",
"connect.mqtt.share.replicate": true,
"tasks.max": 2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions