Integrate NATS client

[tbshrst]

Hello folks,

I'm proposing a NATS integration for kumo to publish messages to NATS servers with JetStream enabled. I think this is a good use case, since NATS is pretty popular, especially in the cloud-native environment. It supports publishing messages and authenticates itself via password or token. Unlike mod-kafka, a builder pattern is used in Rust, so the Lua input must be translated. The proposal employs the async-nats crate.

The interface looks like this:

local nats = kumo.nats.connect {
	servers = {"127.0.0.1:4222"},
	name = "nats-client",
	auth = {
		username = "user",
		password = "somepassword",
	}
}

nats:publish {
	subject = "my.subject",
	payload = "hello from kumomta!",
	headers = {
		["Nats-Msg-Id"] = "unique-key",
	},
	await_ack = true, -- optional
}
nats:close()

I haven't written any documentation yet, as I wanted to wait for your response first.

[wez]

Thanks for this!
I have to confess that I hadn't encountered NATS until now.
I think the code looks reasonable!

We'll need to have some tests in here to verify that things are behaving reasonably.
I'd suggest using https://docs.rs/testcontainers-modules/latest/testcontainers_modules/nats/index.html to spawn a NATS container. Look at crates/integration-tests/src/rabbit.rs to see how we do the rabbitmq tests using testcontainers.

The rabbit tests load the "default" crates/integration-tests/source.lua policy file, which is now getting a bit overloaded with various env vars for different tests, so I'd suggest making a new dedicated test policy file for nats.

Please take a look at these two recent PRs for more general discussion on integration tests to see how to accommodate an alternate policy file for testing:

• #440
• #426

[tbshrst]

Thanks for your suggestions! I have implemented Keysource for credentials, as well as integration tests that test message publishing.