Skip to content

Some interesting Kafka partitioners I've been messing around with

License

Notifications You must be signed in to change notification settings

jeremy-degroot/kafka-partitioners

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 

Repository files navigation

StickyRandomPartitioner

Useful when you have incoming data with a lot of skew, but want to enjoy the benfits of processing similar records together. Records with the same key will go to the same queue for the configured amount of time, then move to a new partition to balance load between consumers of those partitions. Duration is best configured for a multiple of your cache TTL, or the time it takes you to process a batch downstream, or the lifetime of whatever resource you're trying to use.

About

Some interesting Kafka partitioners I've been messing around with

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published