Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow the consumer process to execute Garbage Collection periodically #7

Open
gggeek opened this issue Sep 12, 2019 · 1 comment
Open

Comments

@gggeek
Copy link
Member

gggeek commented Sep 12, 2019

It sometimes happens that php processes leak memory.
This might be in app code, but it might be as well in a deeply nested dependency.
And finding out where the leak comes from is often complex and time consuming; sometimes just impossible.
Sometimes, the leak is not in memory usage but in file handles, or db connections, etc...

So far, the best way to avoid a long-running message consumer process with memory leaks from bringing down the server is to either use a safe value for memory_limit in php.ini, or use the -m cli flag, and then rely on the watchdog to restart the 'killed' process.
We also allow to specify a timeout in number of seconds as well as number of consumed messages (which is not deterministic alas), which helps when we are leaking resources.

An arguably better solution would be to allow a 'garbage collection' routine, made available by the app developer, to be called every N seconds from the message-consumption loop.
This way there would be no need to kill the consumer process, and the cleanup of leaked resources could be much more granular.

@gggeek
Copy link
Member Author

gggeek commented Sep 24, 2019

Note: separately an improvement related to GC has been added to kueueingbundle-sqs ( see https://github.com/kaliop-uk/kueueingbundle-sqs/releases/tag/v0.6.0 )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant