Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unique jobs queue option #26

Open
requaos opened this issue Apr 10, 2019 · 4 comments
Open

Unique jobs queue option #26

requaos opened this issue Apr 10, 2019 · 4 comments

Comments

@requaos
Copy link

requaos commented Apr 10, 2019

Hey I was just tracing around and found this hard fork. Originally I hacked in these two features on top of your work in the soft fork of the go-workers lib, since I like handling errors rather than recovering from panics (which seems to be a common ruby/rails idiom).

I am planning on putting appropriate testing around these two features, but was wondering about getting some informal review of my implementation details.

If you are open to this, and honestly I am not really sure if it's okay because the original developer charges money for these features in his ruby gem, am I infringing on some kinda fuzzy line here?

Moving on, are you willing to give it a look over, and if so should I just open a tentative MR into your soft fork, or should I rebase my changelog on this repo and open an MR here for review?

@requaos requaos changed the title Unique jobs queue option and Throttled worker middleware Unique jobs queue option and Throttled queue middleware Apr 12, 2019
@stefannegrea
Copy link
Collaborator

stefannegrea commented Apr 16, 2019

Hello Neil! We always welcome PRs against this repository and love community contributions. Please note this repository is licensed under MIT and your contributions will have to be licensed under the same terms. All contributions regardless of origin go through the same review process and we will gladly review your PRs too.

Without seeing the code, looks like you have two different and distinct features. Both would be very good additions to this framework. I recommend opening two PRs so we can review the changes independently.

@requaos
Copy link
Author

requaos commented Apr 19, 2019

Let us start with the unique jobs feature then: #27

@requaos
Copy link
Author

requaos commented Apr 19, 2019

The middlewares have been completely redesigned, where before the middleware just needed to satisfy an interface. I am refactoring the code I wrote for a throttled queue, because now the middlewares are pure functions. This feature relies on a ticker running in a separate goroutine. I am reconsidering the storage of this ticker, as it may end up in the worker and no longer be implemented as a middleware

@requaos
Copy link
Author

requaos commented Apr 30, 2019

For the throttled worker, I want to implement it as the number of executions per interval of time with consideration of concurrency selection. I am also considering a distribution option, which would be burst or smooth, where the worker exhausts its execution limit on a tick event or where the tick interval is divided across the number of executions so that the executions occur evenly throughout the period. Any thoughts?

@requaos requaos changed the title Unique jobs queue option and Throttled queue middleware Unique jobs queue option Mar 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants