Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"cluster-topic-manipulation-service" issue #348

Open
nickstatka777 opened this issue Jul 23, 2021 · 1 comment
Open

"cluster-topic-manipulation-service" issue #348

nickstatka777 opened this issue Jul 23, 2021 · 1 comment

Comments

@nickstatka777
Copy link

Hello team, I faced the issue with cluster-topic-manipulation-service it doesn't work for me at all.
So when I'm configuring this it doesn't create a topic for iterations and didn't send any info into "reporter-service" and to "statsd-service" metrics accordingly. However, the other configuration works as it should. Could you please check my config file, maybe I made some mistakes:

{
    "single-cluster-monitor": {
        "class.name": "com.linkedin.xinfra.monitor.apps.SingleClusterMonitor",
        "topic": "xinfra-monitor-topic",
        "zookeeper.connect": "localhost:2181",
        "bootstrap.servers": "localhost:9093",
        "request.timeout.ms": 3000,
        "produce.record.delay.ms": 1000,
        "topic-management.topicManagementEnabled": true,
        "topic-management.topicCreationEnabled": true,
        "topic-management.replicationFactor" : 3,
        "topic-management.partitionsToBrokersRatio" : 2.0,
        "topic-management.rebalance.interval.ms" : 600000,
        "topic-management.preferred.leader.election.check.interval.ms" : 300000,
        "topic-management.topicAddPartitionEnabled": "true",
        "topic-management.topicReassignPartitionAndElectLeaderEnabled": true,
        "client.id": "xinfra-monitor-adminclient",
        "security.protocol": "SSL",
        "ssl.key.password": "${PASSWORD}",
        "ssl.keystore.location": "keystore.p12",
        "ssl.keystore.password": "${PASSWORD}",
        "ssl.keystore.type": "PKCS12",
        "ssl.truststore.location": "truststore.p12",
        "ssl.truststore.password": "${PASSWORD}",
        "ssl.truststore.type": "PKCS12",
        "produce.producer.props": {
            "class.name": "com.linkedin.xinfra.monitor.producer.NewProducer",
            "client.id": "xinfra-monitor-producer",
            "security.protocol": "SSL",
            "ssl.keystore.location": "keystore.p12",
            "ssl.keystore.password": "${PASSWORD}",
            "ssl.keystore.type": "PKCS12",
            "ssl.truststore.location": "truststore.p12",
            "ssl.truststore.password": "${PASSWORD}",
            "ssl.truststore.type": "PKCS12"
        },
        "consume.latency.sla.ms": "20000",
        "consume.consumer.props": {
            "group.id": "a-group-id-mb-888",
            "class.name": "com.linkedin.kmf.consumer.NewConsumer",
            "client.id": "xinfra-monitor-consumer",
            "security.protocol": "SSL",
            "ssl.key.password": "${PASSWORD}",
            "ssl.keystore.location": "keystore.p12",
            "ssl.keystore.password": "${PASSWORD}",
            "ssl.keystore.type": "PKCS12",
            "ssl.truststore.location": "truststore.p12",
            "ssl.truststore.password": "${PASSWORD}",
            "ssl.truststore.type": "PKCS12"
            }
        },
        "jolokia-service": {
            "class.name": "com.linkedin.xinfra.monitor.services.JolokiaService"
        },
        "cluster-topic-manipulation-service":{
        "class.name":"com.linkedin.xinfra.monitor.services.ClusterTopicManipulationService",
        "zookeeper.connect": "localhost:2181",
        "bootstrap.servers":"localhost:9093",
        "topic": "xinfra-monitor-manipulation-topic"
        },
        "offset-commit-service": {
            "class.name": "com.linkedin.xinfra.monitor.services.OffsetCommitService",
            "zookeeper.connect": "localhost:2181",
            "bootstrap.servers": "localhost:9093",
            "consumer.props": {
                "group.id": "target-consumer-group"
            }
        },
        "reporter-service": {
            "class.name": "com.linkedin.xinfra.monitor.services.DefaultMetricsReporterService",
            "report.interval.sec": 1,
            "report.metrics.list": [
				...
            ]
        },
        "statsd-service": {
            "class.name": "com.linkedin.xinfra.monitor.services.StatsdMetricsReporterService",
            "report.statsd.host": "127.0.0.1",
            "report.statsd.port": "9125",
            "report.statsd.prefix": "xinfra-monitor",
            "report.interval.sec": 1,
            "report.metrics.list": [
				...
            ]   
        }
    }    

In addition, I have a question, is it possible that the "cluster-topic-manipulation-service" would work with Kafka brokers only, without zookeepers? This is very important for our infrastructure.
Also, I noticed that there is no possibility to reload service configuration without application restart. Our case is necessary for pkcs12 Keystore reloading every 24 hours. Could you please add this possibility in future releases?

@github-actions
Copy link

This is your first issue in the repository. Thank you for raising this issue.' first issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant