Useful when you have incoming data with a lot of skew, but want to enjoy the benfits of processing similar records together. Records with the same key will go to the same queue for the configured amount of time, then move to a new partition to balance load between consumers of those partitions. Duration is best configured for a multiple of your cache TTL, or the time it takes you to process a batch downstream, or the lifetime of whatever resource you're trying to use.
-
Notifications
You must be signed in to change notification settings - Fork 0
Some interesting Kafka partitioners I've been messing around with
License
jeremy-degroot/kafka-partitioners
About
Some interesting Kafka partitioners I've been messing around with
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published