Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests can be overly resource hungry or otherwise get out of control - add hardening somehow? #30

Open
Cervator opened this issue Nov 28, 2020 · 2 comments

Comments

@Cervator
Copy link
Member

Maybe mildly similar to #25 but distinct - in this case testExample literally went out of memory:

http://jenkins.terasology.io/teraorg/job/Terasology/job/Modules/job/M/job/ModuleTestingEnvironment/job/PR-29/1/console

I'm not sure what the best way to address this is. It doesn't look like the build agent itself ran out of memory, it had a little more space allocated, although maybe if Java requested a large enough chunk of memory (more than 300MB in one reqest?) that could be related

image

Maybe it is something more specific? The build log talks about "GC overhead limit exceeded"

DeprecationTest > testExample() STANDARD_ERROR
    Exception in thread "Chunk-Processing-Reactor" java.lang.OutOfMemoryError: GC overhead limit exceeded
    Exception in thread "Thread-148" java.lang.OutOfMemoryError: GC overhead limit exceeded

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Chunk-Processing-6"
@DarkWeird
Copy link
Contributor

I suspect it is not terminated engines during all tests run. Something with shutdowning.
Every client engine takes 600MB+ by chunks (default view distance and server and client sides)

@Cervator
Copy link
Member Author

Yeah I tried to add more memory to the build agent but it still failed eventually. On the plus side we now have the watchful http://jenkins.terasology.io/teraorg/job/Utilities/job/BuildWatchinator/ which will find and terminate such stalled builds. So at least they won't stall forever anymore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants