Skip to content

Rework or remove CommitRecovery #623

@LMnet

Description

@LMnet

Recently I have worked with the CommitRecovery functionality. And I found that this part of the library is not pleasant to work with.

In particular, I faced the next issues:

  1. recoverCommitWith signature has specific bounds on F. And there is no way to add some extra bounds if I want. For example, I wanted to use log4cats and add some details about retrying inside recoverCommitWith. And at the current moment, I can't that because recoverCommitWith doesn't have a Sync bound. Overall recoverCommitWith's signature feels weird and limiting.
  2. CommitRecovery.Default is pretty nice, but not configurable.
  3. I'm using pureconfig for my configuration and I had to create a custom wrapper around CommitRecovery to make it works nicely with the pureconfig.

Also, It's debatable that this functionality should be built-in inside fs2-kafka. For me, it looks like all this retries stuff could be delegated to a specialized solution like cats-retry. And it may look more natural and composable. ATM this part is not composable: I had to overcome some difficulties to use CommitRecovery with the pureconfig, log4cats, and cats-retry.

I can suggest 2 ways of solving this:

  1. Completely remove this functionality. If the user will want to retry commit failures he can use any library and add retries on top of the commit method. As a bonus point here we will be able to remove Jitter from the library, because it uses only in CommitRecovery logic.
  2. Somehow rework this functionality. Maybe remove CommitRecovery logic from the KafkaConsumerActor and offer a CommitRecovery as an additional option on top of a commit method?

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions