Request for Performance Benchmark Data

[gunli]

Summary

I am currently using the confluent-kafka-go library for a project and would like to understand its performance characteristics better. Specifically, I am interested in any available performance benchmark data or metrics that can help me evaluate the library's suitability for my use case.

I have integrated and tested confluent-kafka-go with my service and found that the performance is not so good as my expectation, the result tells that it can't produce 80000 messages per second, may be only 50000, while sarama can reach 120000 messages easily. Belowed is my config:

kfkCfg := &kafka.ConfigMap{
		"bootstrap.servers":                  strings.Join(cfg.Brokers, ","), 
		"broker.version.fallback":            "2.8.1",                    
		"topic.metadata.refresh.interval.ms": 120000,     
		"acks":             1,                                                  
		"compression.type": "snappy", 
		"go.logs.channel.enable":    true,
		"go.delivery.report.fields": "all", 
		"statistics.interval.ms":    15000,
		"log_level":                 5,     
	}

Details

• Library Version: [2.8.0]
• Environment: [Linux-CentOS, Kafka-2.8.1]
• Use Case: [high-throughput data producer]

Questions

1. Are there any official or community-provided performance benchmarks for confluent-kafka-go?
2. What are the typical throughput and latency metrics for the library under different conditions (e.g., message size, number of partitions, etc.)?
3. Are there any recommended configurations or best practices to achieve optimal performance with confluent-kafka-go?

Additional Information

If there are no existing benchmarks, could you provide guidance on how to set up and conduct performance testing for confluent-kafka-go? Any scripts, tools, or methodologies that you recommend would be greatly appreciated.

Thank you for your assistance.

Best regards,