Working with message acknowledgement in Kafka

[dominikjeske]

I'm trying to understand/configure proper configuration for Kafka but I'm not sure if everything is correctly implemented in Wolverine so below my considerations.

When configuring kafka broker I can setup ProducerConfig with Acks (Acks.None, Acks.Leader, Acks.All) but this would be global configuration for all messages - so when I would like to have different behavior per message type how can I do this?

Another thing is that in docs https://jeremydmiller.com/2023/08/03/understanding-endpoints-in-wolverine-messaging/ you are saying that "Outgoing messages are deleted in the durable outbox upon successful sending acknowledgements from the external broker." But I cannot see any connection between configuration of durable outbox and kafka - so in theory I could have durable outbox but my broker could be still Acks.None because wolverine is not configuring this?

And last thing - message delivery option and envelope has AckRequested but this property is also not used in kafka so setting this will do nothing. In contrast in MQQT for example it is used https://github.com/JasperFx/wolverine/blob/main/src/Transports/MQTT/Wolverine.MQTT/Internals/MqttEnvelopeMapper.cs#L83 so setting this option will change transport behavior. If I'm not missing something I thing that this property should either do something or throw NotSupportedException if this is not implemented.

[jeremydmiller]

@dominikjeske Sorry, didn't mean to let this one slip by me. I'll have to review myself tomorrow and get back to you. Envelope.AckRequested is for Wolverine workflow though. That would be mapped to a header as part of the built in EnvelopeMapper base type. But better answers tomorrow.

[jeremydmiller]

@dominikjeske, finally, try this:

• If you're using a durable endpoint, Wolverine is committing the producer position as soon as the messages are persisted in the inbox, then it's all Wolverine from there.
• Envelope.AckRequested is only relevant to Wolverine internals and that's Wolverine itself sending an Acknowledgement/FailureAcknowledgement

When configuring kafka broker I can setup ProducerConfig with Acks (Acks.None, Acks.Leader, Acks.All) but this would be global configuration for all messages - so when I would like to have different behavior per message type how can I do this?

ProducerConfig is specific to an Endpoint. You could make it so that each message type only goes to a single topic I guess