Service is restarted after rabbitmq_management is enabled

[llussy]

When enable the rabbitmq_management plugin,rabbitmq service will always restart.
I have five machines, two of them have problems, the same installation steps.
Return to normal after disable rabbitmq_management.

rabbitmq: 3.7.7
erlang: 21.0-1 
system version： CentOS Linux release 7.6.1810 (Core) 

I check in the official documentation, erlang and rabbitmq are compatible, I suspect it is related to the following information, but I don't have a relevant solution.
kernel: 1_scheduler[28839]: segfault at 210 ip 0000000000610d64 sp 00007f36aabfe900 error 4 in beam.smp[400000+345000]
DNS resolution of the hostname in /etc/hosts.

rabbit error log:

2021-05-20 14:02:16.524 [info] <0.7.0> Log file opened with Lager
2021-05-20 14:02:17.085 [info] <0.275.0> 
 Starting RabbitMQ 3.7.7 on Erlang 21.0
 Copyright (C) 2007-2018 Pivotal Software, Inc.
 Licensed under the MPL.  See http://www.rabbitmq.com/
2021-05-20 14:02:17.085 [info] <0.275.0> 
 node           : rabbit@advert_middleware_128v24_qcloud
 home dir       : /var/lib/rabbitmq
 config file(s) : (none)
 cookie hash    : FA++IKNKvR5XsFHfGX4Cfg==
 log(s)         : /var/log/rabbitmq/rabbit@advert_middleware_128v24_qcloud.log
                : /var/log/rabbitmq/rabbit@advert_middleware_128v24_qcloud_upgrade.log
 database dir   : /var/lib/rabbitmq/mnesia/rabbit@advert_middleware_128v24_qcloud
2021-05-20 14:02:17.657 [info] <0.283.0> Memory high watermark set to 6353 MiB (6662466764 bytes) of 15884 MiB (16656166912 bytes) total
2021-05-20 14:02:17.661 [info] <0.285.0> Enabling free disk space monitoring
2021-05-20 14:02:17.661 [info] <0.285.0> Disk free limit set to 50MB
2021-05-20 14:02:17.664 [info] <0.287.0> Limiting to approx 924 file handles (829 sockets)
2021-05-20 14:02:17.664 [info] <0.288.0> FHC read buffering:  OFF
2021-05-20 14:02:17.664 [info] <0.288.0> FHC write buffering: ON
2021-05-20 14:02:17.665 [info] <0.275.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2021-05-20 14:02:17.698 [info] <0.275.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2021-05-20 14:02:17.699 [info] <0.275.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
2021-05-20 14:02:17.699 [info] <0.275.0> Priority queues enabled, real BQ is rabbit_variable_queue
2021-05-20 14:02:17.703 [info] <0.314.0> Starting rabbit_node_monitor
2021-05-20 14:02:17.726 [info] <0.275.0> Management plugin: using rates mode 'basic'
2021-05-20 14:02:17.727 [info] <0.346.0> Making sure data directory '/var/lib/rabbitmq/mnesia/rabbit@advert_middleware_128v24_qcloud/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
2021-05-20 14:02:17.731 [info] <0.346.0> Starting message stores for vhost '/'
2021-05-20 14:02:17.731 [info] <0.350.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
2021-05-20 14:02:17.732 [info] <0.346.0> Started message store of type transient for vhost '/'
2021-05-20 14:02:17.732 [info] <0.353.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
2021-05-20 14:02:17.732 [warning] <0.353.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch
2021-05-20 14:02:17.733 [info] <0.346.0> Started message store of type persistent for vhost '/'
2021-05-20 14:02:17.735 [info] <0.388.0> started TCP Listener on [::]:5672
2021-05-20 14:02:17.735 [info] <0.275.0> Setting up a table for connection tracking on this node: tracked_connection_on_node_rabbit@advert_middleware_128v24_qcloud
2021-05-20 14:02:17.735 [info] <0.275.0> Setting up a table for per-vhost connection counting on this node: tracked_connection_per_vhost_on_node_rabbit@advert_middleware_128v24_qcloud
2021-05-20 14:02:17.761 [info] <0.439.0> Management plugin started. Port: 15672
2021-05-20 14:02:17.761 [info] <0.545.0> Statistics database started.
2021-05-20 14:02:17.791 [notice] <0.109.0> Changed loghwm of /var/log/rabbitmq/rabbit@advert_middleware_128v24_qcloud.log to 50
2021-05-20 14:02:17.840 [info] <0.7.0> Server startup complete; 3 plugins started.
 * rabbitmq_management
 * rabbitmq_management_agent
 * rabbitmq_web_dispatch
2021-05-20 14:05:35.761 [info] <0.7.0> Log file opened with Lager
2021-05-20 14:05:36.282 [info] <0.275.0> 
 Starting RabbitMQ 3.7.7 on Erlang 21.0
 Copyright (C) 2007-2018 Pivotal Software, Inc.
 Licensed under the MPL.  See http://www.rabbitmq.com/
2021-05-20 14:05:36.282 [info] <0.275.0> 
 node           : rabbit@advert_middleware_128v24_qcloud
 home dir       : /var/lib/rabbitmq
 config file(s) : (none)
 cookie hash    : FA++IKNKvR5XsFHfGX4Cfg==
 log(s)         : /var/log/rabbitmq/rabbit@advert_middleware_128v24_qcloud.log
                : /var/log/rabbitmq/rabbit@advert_middleware_128v24_qcloud_upgrade.log
 database dir   : /var/lib/rabbitmq/mnesia/rabbit@advert_middleware_128v24_qcloud
2021-05-20 14:05:36.832 [info] <0.283.0> Memory high watermark set to 6353 MiB (6662466764 bytes) of 15884 MiB (16656166912 bytes) total
2021-05-20 14:05:36.836 [info] <0.285.0> Enabling free disk space monitoring
2021-05-20 14:05:36.836 [info] <0.285.0> Disk free limit set to 50MB
2021-05-20 14:05:36.838 [info] <0.287.0> Limiting to approx 924 file handles (829 sockets)
2021-05-20 14:05:36.838 [info] <0.288.0> FHC read buffering:  OFF
2021-05-20 14:05:36.839 [info] <0.288.0> FHC write buffering: ON
2021-05-20 14:05:36.839 [info] <0.275.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2021-05-20 14:05:36.878 [info] <0.275.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2021-05-20 14:05:36.878 [info] <0.275.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
2021-05-20 14:05:36.879 [info] <0.275.0> Priority queues enabled, real BQ is rabbit_variable_queue
2021-05-20 14:05:36.892 [info] <0.314.0> Starting rabbit_node_monitor
2021-05-20 14:05:36.914 [info] <0.275.0> Management plugin: using rates mode 'basic'
2021-05-20 14:05:36.915 [info] <0.346.0> Making sure data directory '/var/lib/rabbitmq/mnesia/rabbit@advert_middleware_128v24_qcloud/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
2021-05-20 14:05:36.918 [info] <0.346.0> Starting message stores for vhost '/'
2021-05-20 14:05:36.918 [info] <0.350.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
2021-05-20 14:05:36.919 [info] <0.346.0> Started message store of type transient for vhost '/'
2021-05-20 14:05:36.919 [info] <0.353.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
2021-05-20 14:05:36.919 [warning] <0.353.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch
2021-05-20 14:05:36.920 [info] <0.346.0> Started message store of type persistent for vhost '/'
2021-05-20 14:05:36.922 [info] <0.388.0> started TCP Listener on [::]:5672
2021-05-20 14:05:36.922 [info] <0.275.0> Setting up a table for connection tracking on this node: tracked_connection_on_node_rabbit@advert_middleware_128v24_qcloud
2021-05-20 14:05:36.922 [info] <0.275.0> Setting up a table for per-vhost connection counting on this node: tracked_connection_per_vhost_on_node_rabbit@advert_middleware_128v24_qcloud
2021-05-20 14:05:36.947 [info] <0.439.0> Management plugin started. Port: 15672
2021-05-20 14:05:36.947 [info] <0.545.0> Statistics database started.
2021-05-20 14:05:36.977 [notice] <0.109.0> Changed loghwm of /var/log/rabbitmq/rabbit@advert_middleware_128v24_qcloud.log to 50
2021-05-20 14:05:37.045 [info] <0.7.0> Server startup complete; 3 plugins started.
 * rabbitmq_management
 * rabbitmq_management_agent
 * rabbitmq_web_dispatch

message error log:

May 20 14:02:05 advert_middleware_128v24_qcloud kernel: 1_scheduler[28839]: segfault at 210 ip 0000000000610d64 sp 00007f36aabfe900 error 4 in beam.smp[400000+345000]
May 20 14:02:05 advert_middleware_128v24_qcloud systemd: rabbitmq-server.service: main process exited, code=killed, status=11/SEGV
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: Error: unable to perform an operation on node 'rabbit@advert_middleware_128v24_qcloud'. Please see diagnostics information and suggestions below.
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: Most common reasons for this are:
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues)
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server)
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * Target node is not running
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: In addition to the diagnostics info below:
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * See the CLI, clustering and networking guides on http://rabbitmq.com/documentation.html to learn more
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * Consult server logs on node rabbit@advert_middleware_128v24_qcloud
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: DIAGNOSTICS
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: ===========
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: attempted to contact: [rabbit@advert_middleware_128v24_qcloud]
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: rabbit@advert_middleware_128v24_qcloud:
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * connected to epmd (port 4369) on advert_middleware_128v24_qcloud
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * epmd reports: node 'rabbit' not running at all
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: no other nodes on advert_middleware_128v24_qcloud
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * suggestion: start the node
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: Current node details:
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * node name: rabbitmqcli69@advert_middleware_128v24_qcloud
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * effective user's home directory: /var/lib/rabbitmq
May 20 14:02:05 advert_middleware_128v24_qcloud rabbitmqctl: * Erlang cookie hash: FA++IKNKvR5XsFHfGX4Cfg==
May 20 14:02:05 advert_middleware_128v24_qcloud systemd: Unit rabbitmq-server.service entered failed state.
May 20 14:02:05 advert_middleware_128v24_qcloud systemd: rabbitmq-server.service failed.
May 20 14:02:15 advert_middleware_128v24_qcloud systemd: rabbitmq-server.service holdoff time over, scheduling restart.
May 20 14:02:15 advert_middleware_128v24_qcloud systemd: Cannot add dependency job for unit rpcbind.socket, ignoring: Unit not found.
May 20 14:02:15 advert_middleware_128v24_qcloud systemd: Stopped RabbitMQ broker.
May 20 14:02:15 advert_middleware_128v24_qcloud systemd: Starting RabbitMQ broker...
May 20 14:02:17 advert_middleware_128v24_qcloud rabbitmq-server: ##  ##
May 20 14:02:17 advert_middleware_128v24_qcloud rabbitmq-server: ##  ##      RabbitMQ 3.7.7. Copyright (C) 2007-2018 Pivotal Software, Inc.
May 20 14:02:17 advert_middleware_128v24_qcloud rabbitmq-server: ##########  Licensed under the MPL.  See http://www.rabbitmq.com/
May 20 14:02:17 advert_middleware_128v24_qcloud rabbitmq-server: ######  ##
May 20 14:02:17 advert_middleware_128v24_qcloud rabbitmq-server: ##########  Logs: /var/log/rabbitmq/rabbit@advert_middleware_128v24_qcloud.log
May 20 14:02:17 advert_middleware_128v24_qcloud rabbitmq-server: /var/log/rabbitmq/rabbit@advert_middleware_128v24_qcloud_upgrade.log
May 20 14:02:17 advert_middleware_128v24_qcloud rabbitmq-server: Starting broker...
May 20 14:02:17 advert_middleware_128v24_qcloud rabbitmq-server: systemd unit for activation check: "rabbitmq-server.service"
May 20 14:02:17 advert_middleware_128v24_qcloud systemd: Started RabbitMQ broker.

[michaelklishin]

I will convert this issue to a GitHub discussion. Currently GitHub will automatically close and lock the issue even though your question will be transferred and responded to elsewhere. This is to let you know that we do not intend to ignore this but this is how the current GitHub conversion mechanism makes it seem for the users :(

[michaelklishin]

is the line you are looking for. The runtime seems to segfault. Upgrading to 3.7.27 (at least) and Erlang to the latest version compatible with 3.7 is the only suggestion we have.

RabbitMQ 3.7 is out of support and 3.7.7 specifically is 20 over 30 releases behind. Consider upgrading.

[llussy]

Thank you, I will try to upgrade 3.7.27。

[llussy]

rabbitmq upgrade to 3.7.27,erlang upgrade to 21.3.8.9. after rabbitmq_management service normal work. thank you.