Worker container processes reach PHP memory limit and crash

[graham-web]

Trying to debug a small percentage of messages not being sent...

I'm noticing a lot of the following error in the logs for my "worker" container:

PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 335872 bytes) in /var/www/html/vendor/doctrine/dbal/src/Driver/PDO/Statement.php on line 130

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 335872 bytes) in /var/www/html/vendor/doctrine/dbal/src/Driver/PDO/Statement.php on line 130

In Statement.php line 130:

Error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 335872 bytes)

messenger:consume [-l|--limit LIMIT] [-f|--failure-limit FAILURE-LIMIT] [-m|--memory-limit MEMORY-LIMIT] [-t|--time-limit TIME-LIMIT] [--sleep SLEEP] [-b|--bus BUS] [--queues QUEUES] [--no-reset] [--] [...]
2025-05-01 13:43:01,720 WARN exited: messenger-consume-email_00 (exit status 255; not expected)
2025-05-01 13:43:02,747 INFO spawned: 'messenger-consume-email_00' with pid 332

Should the default supervisor command line use the --memory-limit option to prevent this?

[henmohr]

Hey thanks for your reporting.

A hyphothesis is that messenger:consume is spawning multiple times OR your system is out of memory.
Can you pass the parameter --time-limit=50 and see if the error persists?

Yes, i guess limiting memory usage it's a good practice too.

Could you test and report back here?

Thanks!

[graham-web]

The system isn't running out of memory, and the container keeps running. The log snippet I shared shows the messenger process for the email queue hitting the PHP memory_limit, then supervisor respawning it.

$ php -i | grep memory_limit
memory_limit => 512M => 512M

While sending to a newsletter of ~3000 today, this happened 6 times, approximately 90 seconds between each crash.

Just to note, here is Symfony's documentation on configuring this for production
https://symfony.com/doc/5.x/messenger.html#deploying-to-production

I've put --memory_limit=256M on this instance for the moment. Waiting to see what happens when the next message goes out next week.

[O-Mutt]

Hey thanks for your reporting.

A hyphothesis is that messenger:consume is spawning multiple times OR your system is out of memory. Can you pass the parameter --time-limit=50 and see if the error persists?

Yes, i guess limiting memory usage it's a good practice too.

Could you test and report back here?

Thanks!

That should only occur if the messenger:consume is in the cron and not the worker. The more recent image should prevent the first case

[cibero42]

Hello @graham-web ,

Could you please do a docker compose pull to ensure that you are running the latest build?

Furthermore, could you please mention which tag you are using?

[christoph93]

I'm facing a similar issue on 5.2.5-apache.

mautic-cron    | Rebuilding contacts for segment 7
mautic-cron    | 0 total contact(s) to be added in batches of 300
mautic-cron    | 0 total contact(s) to be removed in batches of 300
mautic-cron    | 0 contact(s) affected
mautic-cron    | Rebuilding contacts for segment 17
mautic-cron    | 0 total contact(s) to be added in batches of 300
mautic-cron    | 0 total contact(s) to be removed in batches of 300
mautic-cron    | 0 contact(s) affected
mautic-web     | 127.0.0.1 - - [11/May/2025:02:23:05 +0000] "GET / HTTP/1.1" 302 665 "-" "curl/7.88.1"
mautic-cron    | [11-May-2025 02:23:05 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/docroot/app/bundles/LeadBundle/Entity/CustomFieldRepositoryTrait.php on line 100
mautic-cron    | 
mautic-cron    | Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/docroot/app/bundles/LeadBundle/Entity/CustomFieldRepositoryTrait.php on line 100
mautic-cron    | [11-May-2025 02:23:05 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 32768 bytes) in /var/www/html/vendor/monolog/monolog/src/Monolog/DateTimeImmutable.php on line 1
mautic-cron    | 
mautic-cron    | Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 32768 bytes) in /var/www/html/vendor/monolog/monolog/src/Monolog/DateTimeImmutable.php on line 1

I've set the memory_limit to 4094M on a VPS with 8gb of RAM . Is there something else I can try?

[O-Mutt]

I'm facing a similar issue on 5.2.5-apache.

mautic-cron    | Rebuilding contacts for segment 7
mautic-cron    | 0 total contact(s) to be added in batches of 300
mautic-cron    | 0 total contact(s) to be removed in batches of 300
mautic-cron    | 0 contact(s) affected
mautic-cron    | Rebuilding contacts for segment 17
mautic-cron    | 0 total contact(s) to be added in batches of 300
mautic-cron    | 0 total contact(s) to be removed in batches of 300
mautic-cron    | 0 contact(s) affected
mautic-web     | 127.0.0.1 - - [11/May/2025:02:23:05 +0000] "GET / HTTP/1.1" 302 665 "-" "curl/7.88.1"
mautic-cron    | [11-May-2025 02:23:05 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/docroot/app/bundles/LeadBundle/Entity/CustomFieldRepositoryTrait.php on line 100
mautic-cron    | 
mautic-cron    | Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/docroot/app/bundles/LeadBundle/Entity/CustomFieldRepositoryTrait.php on line 100
mautic-cron    | [11-May-2025 02:23:05 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 32768 bytes) in /var/www/html/vendor/monolog/monolog/src/Monolog/DateTimeImmutable.php on line 1
mautic-cron    | 
mautic-cron    | Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 32768 bytes) in /var/www/html/vendor/monolog/monolog/src/Monolog/DateTimeImmutable.php on line 1

I've set the memory_limit to 4094M on a VPS with 8gb of RAM . Is there something else I can try?

Could you give us some more info on what your setup is? Understanding what crons are running would be helpful here too. A command like docker compose exec -u www-data -w /var/www/html mautic_cron crontab -l should print out what is supposed to be running

[christoph93]

@O-Mutt these are cron jobs. I believe they're the default ones that come with the docker image.

* * * * * php /var/www/html/bin/console mautic:segments:update 2>&1 | tee /tmp/stdout
* * * * * php /var/www/html/bin/console mautic:campaigns:update 2>&1 | tee /tmp/stdout
* * * * * php /var/www/html/bin/console mautic:campaigns:trigger 2>&1 | tee /tmp/stdout
* * * * * php /var/www/html/bin/console mautic:contacts:scheduled_export 2>&1 | tee /tmp/stdout

[cibero42]

@O-Mutt these are cron jobs. I believe they're the default ones that come with the docker image.

* * * * * php /var/www/html/bin/console mautic:segments:update 2>&1 | tee /tmp/stdout
* * * * * php /var/www/html/bin/console mautic:campaigns:update 2>&1 | tee /tmp/stdout
* * * * * php /var/www/html/bin/console mautic:campaigns:trigger 2>&1 | tee /tmp/stdout
* * * * * php /var/www/html/bin/console mautic:contacts:scheduled_export 2>&1 | tee /tmp/stdout

Hi @christoph93 ! You don't seem to be running the latest build from Mautic

Could you please ensure that you're running the tag 5-apache across containers and execute docker container pull?

Thanks!

[christoph93]

Thanks for your reply @cibero42 , I'm now using 5-apache but it didn't fix the issue:

mautic-cron    | 0 total contact(s) to be removed in batches of 300
mautic-cron    | 
mautic-cron    | 
mautic-web     | 172.19.0.3 - - [26/May/2025:02:13:02 +0000] "GET /media/images/logo.png HTTP/1.1" 200 1128812 "-" "-"
mautic-cron    |   9/21 [============>---------------]  42%[26-May-2025 02:13:02 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 44625859 bytes) in /var/www/html/vendor/doctrine/dbal/src/Driver/PDO/Statement.php on line 130
mautic-cron    | 
mautic-cron    | Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 44625859 bytes) in /var/www/html/vendor/doctrine/dbal/src/Driver/PDO/Statement.php on line 130
mautic-cron    | 
mautic-cron    | In Statement.php line 130:
mautic-cron    |                                                                                
mautic-cron    |   Error: Allowed memory size of 134217728 bytes exhausted (tried to allocate   
mautic-cron    |   44625859 bytes)                                                              
mautic-cron    |                                                                                
mautic-cron    | 
mautic-cron    | mautic:campaigns:trigger [-i|--campaign-id [CAMPAIGN-ID]] [--campaign-limit [CAMPAIGN-LIMIT]] [--contact-id [CONTACT-ID]] [--contact-ids [CONTACT-IDS]] [--min-contact-id [MIN-CONTACT-ID]] [--max-contact-id [MAX-CONTACT-ID]] [--thread-id [THREAD-ID]] [--max-threads [MAX-THREADS]] [--kickoff-only] [--scheduled-only] [--inactive-only] [-l|--batch-limit [BATCH-LIMIT]] [-d|--exclude [EXCLUDE]] [--bypass-locking] [-t|--timeout TIMEOUT] [-x|--lock_mode LOCK_MODE] [-f|--force]
mautic-cron    | 
mautic-web     | [Mon May 26 02:13:04.797838 2025] [php:notice] [pid 114:tid 114] [client 127.0.0.1:36936] PHP Deprecation - Creation of dynamic property Mautic\\Middleware\\MiddlewareBuilder::$specs is deprecated - in file /var/www/html/docroot/app/middlewares/MiddlewareBuilder.php - at line 24
mautic-web     | 127.0.0.1 - - [26/May/2025:02:13:04 +0000] "GET / HTTP/1.1" 302 665 "-" "curl/7.88.1"
mautic-cron    | [26-May-2025 02:13:05 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/docroot/app/bundles/LeadBundle/Entity/CustomFieldRepositoryTrait.php on line 100
mautic-cron    | 
mautic-cron    | Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/docroot/app/bundles/LeadBundle/Entity/CustomFieldRepositoryTrait.php on line 100
mautic-cron    | 
mautic-cron    | [26-May-2025 02:13:05 UTC] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 32768 bytes) in /var/www/html/vendor/symfony/string/UnicodeString.php on line 1
mautic-cron    | 
mautic-cron    | Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 32768 bytes) in /var/www/html/vendor/symfony/string/UnicodeString.php on line 1
mautic-web     | [Mon May 26 02:13:09.933269 2025] [php:notice] [pid 29:tid 29] [client 127.0.0.1:36946] PHP Deprecation - Creation of dynamic property Mautic\\Middleware\\MiddlewareBuilder::$specs is deprecated - in file /var/www/html/docroot/app/middlewares/MiddlewareBuilder.php - at line 24
mautic-web     | 127.0.0.1 - - [26/May/2025:02:13:09 +0000] "GET / HTTP/1.1" 302 665 "-" "curl/7.88.1"
mautic-web     | [Mon May 26 02:13:15.061555 2025] [php:notice] [pid 30:tid 30] [client 127.0.0.1:35388] PHP Deprecation - Creation of dynamic property Mautic\\Middleware\\MiddlewareBuilder::$specs is deprecated - in file /var/www/html/docroot/app/middlewares/MiddlewareBuilder.php - at line 24
mautic-web     | 127.0.0.1 - - [26/May/2025:02:13:15 +0000] "GET / HTTP/1.1" 302 665 "-" "curl/7.88.1"

Now the logs are also filled with the deprecated message.

Still showing 4gb allowed memory

root@b3e1cb32555d:/var/www/html/docroot# php -ini | grep memory
memory_limit => 4096M => 4096M

Could this have something to do with image sizes? We're using fairly large images in e-mails and the memory limit error show right after it gets the logo.png.