Description
Hello, I have some questions when using lua-resty-lua. I wanted to avoid having to re-connect to kafka client every time a request came into Lua code, so I used something like singleton mode to make a kafka client connection and then use the same connection to send messages to kafka. I'd like to ask if this is necessary?
Here's connection code.
KafkaUtils = {}
local mt = { __index = KafkaUtils }
-- 基础类方法 new
function KafkaUtils.new()
local broker_list = {
{host = "172.X.X.1", port = 9092},
{host = "172.X.X.2", port = 9092},
{host = "172.X.X.3", port = 9092}
}
kafkaProducer = producer:new(broker_list, { producer_type = "async", refresh_interval = 10000, required_acks = 0})
return setmetatable({kafkaProducer = kafkaProducer}, mt)
end
--- 仿单例模式
function KafkaUtils:Instance()
if self.instance == nil then
self.instance = self:new()
end
return self.instance
end
--- 生产者发送消息
function KafkaUtils:sendMsg(topic, message)
local kafkaProducer = self.kafkaProducer;
local ok, err = kafkaProducer:send(topic, nil, message);
if not ok then
ngx.log(ngx.ERR, 'kafka send err:', err)
end
end
return KafkaUtils;Here's code for sending massage.
--- 发送消息
KafkaUtils:Instance():sendMsg(topic, massage);I currently have 12 servers using openrsty for message production and 3 servers for kafka clusters and 9 partitions per topic. After testing, when the concurrency is around 4000 connections, only 500 messages can be sent per second, and each message is only 30 characters long. I would like to ask how to configure to increase the number of messages sent?
Looking forward to your reply.