Skip to content

Option to use message count as metric in kafka scaler on invalid offset #6775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

DaniilAnichin
Copy link

Provide a description of what has been changed

Checklist

  • [*] When introducing a new scaler, I agree with the scaling governance policy
  • [*] I have verified that my change is according to the deprecations & breaking changes policy
  • [*] Tests have been added
  • [*] Changelog has been updated and is aligned with our changelog requirements
  • [*] A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified) - I did my best to check, but it looks like no changes required
  • [*] A PR is opened to update the documentation on (repo) (if applicable)
  • [*] Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #

Relates to keda-docs PR

@DaniilAnichin DaniilAnichin requested a review from a team as a code owner May 14, 2025 08:25
@DaniilAnichin DaniilAnichin force-pushed the feature/kafka-beginning-offset branch from 3791ea8 to d303e54 Compare May 14, 2025 08:39
@DaniilAnichin
Copy link
Author

DaniilAnichin commented May 14, 2025

Hi, I did my best to prepare the PR in the appropriate way (signed commits, added tests, added docs PR)
I'm not very good with go though, and I'm pretty sure my code changes itself are suboptimal.
I think it would be best to adjust the calls in such a way that earliest offsets are only loaded when the flag is set, but I did not get how exactly to do so, since it's done on the outside and passed as parameter, so we would need to type it as (some data or null) or something that direction.
I would love if someone close to the kafka scalers / project in general would advise on should I optimize it or it's good enough as is

Also I didn't touch experimental kafka-go scaler, to minimize back-n-force (assumed it would be better to polish one first, than copy to the second one if required)

P.S.: For what it's worth, I did build this version locally and deployed to the cluster, and it seems to work as expected so far

@DaniilAnichin
Copy link
Author

Based on other PRs, I'll tag @JorTurFer , @zroubalik and @dttung2905
Excuse me if this is exessive

@DaniilAnichin DaniilAnichin force-pushed the feature/kafka-beginning-offset branch from d303e54 to e879450 Compare May 14, 2025 09:33
@JorTurFer
Copy link
Member

The code looks correct, but I don't know anything about kafka, @dttung2905 or @zroubalik PTAL

Copy link
Contributor

@dttung2905 dttung2905 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So sorry for the late reply. I have put a 👀 icon on it and was actually reviewing the PR. But for some reason I left half way and totally forget about it 😞
LGTM too !

@DaniilAnichin
Copy link
Author

/run-e2e kafka

@dttung2905
Copy link
Contributor

dttung2905 commented Jun 1, 2025

/run-e2e kafka
Update: You can check the progress here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants