Skip to content

Bridge: use batch count=0 to signal backoff, not "done" #1369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

svix-onelson
Copy link
Contributor

Looks like the done field doesn't signify we're at the end of the topic like I originally thought.

Using the count as a proxy for this works well enough for now, but may not in the future if we expose filters like channels/event types.

Follow-on from #1363

Looks like the `done` field doesn't signify we're at the end of the
topic like I originally thought.

Using the count as a proxy for this works well enough for now, but may
not in the future if we expose filters like channels/event types.

Follow-on from #1363
@svix-onelson svix-onelson requested a review from a team as a code owner July 12, 2024 03:24
@svix-onelson svix-onelson marked this pull request as draft July 12, 2024 03:31
@svix-onelson
Copy link
Contributor Author

I did check that the field is being set as I expect when we return each iterator and I think the problem I was seeing while testing the poller locally is just that create message rate limiting meant bridge was draining the stream faster than I was filling it.

I do see better throughput by using count, which makes sense, but I'm also polling much harder using this method. I think we should just leave it where it is for now and I should just raise my own rate limit 😅

@svix-jplatte svix-jplatte deleted the onelson/bridge-poller-backoff branch August 22, 2024 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant