PHP Fatal error: Allowed Memory Size

[aleemb]

On a few occasions where I imagine the stack trace is large, BugsnagLogger is unable to log the error and fails with PHP Fatal error: Allowed Memory Size. This means the original error is lost for good since it isn't logger and the PHP error logs don't end up showing this error either as the request fails fatally.

Some kind of check or config option to circumvent this would be very helpful.

[GrahamCampbell]

Thank you for the report. That is very interesting. Out of interest, what is your PHP memory limit, and are you certain that this is caused by Bugsnag, and not Bugsnag trying to catch the out of memory error, caused by your app?

[aleemb]

Some more details. If I disable BugSnag the error dumps to the screen just fine. However, If I enable BugSnag, I see the following in the logs

2017/04/03 21:22:23 [error] 58#0: *48368 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 262115 bytes) in /var/www/foobar/vendor/bugsnag/bugsnag-psr-logger/src/BugsnagLogger.php on line 48" while reading response header from upstream, client: 192.168.64.1, server: foobar.com, request: "POST /admin/form HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php5.6-fpm.sock:", host: "foobar.com", referrer: "https://foobar.com/admin/form"

and the browser default 500 error page

[aleemb]

I am on v2.4.0 of bugsnag/bugsnag-laravel which is using v3.4.0 of bugsnag/bugsnag

[dominics]

See also #19 etc.

are you certain that this is caused by Bugsnag, and not Bugsnag trying to catch the out of memory error, caused by your app

Both of those cases are problematic (and my logs indicate both occur).

The Bugsnag client allocates significant amounts of memory in many places. Everything from the static array in ErrorTypes allocating 32k, to the json_encode() in HttpClient using essentially unbound memory (I've seen megabytes there).

So, both cases happen:

• sometimes an error just coincidentally happens when memory usage is near the limit and, were it not for Bugsnag's allocations, the error would have been otherwise handled normally
• sometimes something else in the app runs out of memory, and BugSnag tries to report the error and then hits a second OOM error

Both result in nothing in Bugsnag. Obviously the first case is purely caused by BugSnag, but the second is pretty poor form too.

In fact, unless the Bugsnag client checks memory_get_usage(), and optionally issues an ini_set() (to account for the extra memory that's about to be allocated), I don't see how comments like #19 (comment) ("looks like bugsnag-php should be able to catch memory errors") can be correct...

Maybe PHP used to disable the memory limit after the first time it hit it? It certainly doesn't any longer.

[GrahamCampbell]

It is not possible to catch out of memory errors:

Attempting to do anything, such as raising the memory limit (which would probably be a bad idea), will use more memory, and also run out of memory.

[dominics]

@GrahamCampbell A couple of notes:

It's not a good idea to test this bug with a recursive stack overflow. That's because xdebug provides a stack limit that on normal PHP instances will hit much earlier (256 recursions by default?) than a memory limit. So you run the risk of confusing people trying to reproduce in their dev environments. I suggest $v = str_repeat(' ', PHP_MAX_INT); (it's also quicker to hit the limit given).

Next, check out this 3v4l: https://3v4l.org/qIH7Z

register_shutdown_function(function () {
    var_dump(memory_get_usage());
    var_dump(error_get_last());
    
    $lotsOfMemoryLeftThatCanBeUsed = str_repeat(' ', 1000);
    var_dump(strlen($lotsOfMemoryLeftThatCanBeUsed));
});

$v = str_repeat(' ', PHP_INT_MAX);

This works, indicating you can catch memory errors (just not with the exception/error handler). Furthermore, if the allocation that failed was large, you'll have plenty of available memory to process the error.

Even where the OOM is caused by small allocations, there are still approaches (albeit hacky ones) that can deal the error (by reserving "emergency memory"): http://stackoverflow.com/a/27581958/10831

[dominics]

Attempting to do anything, such as raising the memory limit (which would probably be a bad idea), will use more memory

Just wanted to specifically call this out too (although I totally agree that it could be a bad idea unless the user opts in): raising the memory limit does not use more memory.

• Calling ini_set the first time in a process uses about 32 bytes. It's unlikely (to my mind, in practical situations) that the user ran out of memory on an attempted allocation of less than this amount, so the call to ini_set will almost always succeed.
• Calling it a second time in a process doesn't use any memory, and will always succeed, even under OOM conditions.

See, for example, this 3v4l: https://3v4l.org/Xuun2

ini_set('memory_limit', '1M');

register_shutdown_function(function () {
    ini_set('memory_limit', '10M');
    $v = str_repeat(' ', 2 ** 21);
    var_dump(strlen($v));
});

$v = str_repeat(' ', 2 ** 21);

(The first str_repeat causes an OOM error, the second str_repeat allocates the same amount successfully.) So, with that, I think we have all the tools necessary to get this error shipped to Bugsnag successfully, 99% of the time.

[GrahamCampbell]

It is not always to recover, however:

https://3v4l.org/K2Uq4

[GrahamCampbell]

Calling ini_set the first time in a process uses about 32 bytes. It's unlikely (to my mind, in practical situations) that the user ran out of memory on an attempted allocation of less than this amount, so the call to ini_set will almost always succeed.

It's possible people will have disabled that function, or do not want their memory limit messed with.

[dominics]

@GrahamCampbell I think I've addressed both of those comments.

although I totally agree that it could be a bad idea unless the user opts in

etc.

[GrahamCampbell]

It's not a good idea to test this bug with a recursive stack overflow. That's because xdebug provides a stack limit that on normal PHP instances will hit much earlier (256 recursions by default?) than a memory limit.

Xdebug is generally not installed on production servers, where Bugsnag is primarily used. I'd argue it's a bad practice to install xdebug on production machines, if not for the huge performance impact alone.

[dominics]

bad practice to install xdebug on production machines

@GrahamCampbell can you take a bit longer and read what I've written? My next sentence was "trying to reproduce in their dev environments"...

[GrahamCampbell]

I think a memory limit being bust by trying to allocate memory for a function call is actually the most common case in the real world, rather than simulated massive strings, since in that case, php's dropping the string from memory, which is why you are able to continue.

[dominics]

actually the most common case in the real world

I'd probably guess

• buffered MySQL queries using the mysqlnd driver
• HTTP response payloads using curl

would both be more common cases than a stack overflow.

[GrahamCampbell]

The most common cases I see are accidental infinite recursion in Eloquent, or the service container. I mean, they are all valid cases, so. We need to keep them all in mind. :)