systemd circular dependency caused by cloud-init and firewalld

[uniuuu]

Bug report

systemd circular dependency with cloud-init and firewalld
Just heads up that if you have cloud-init and firewalld you'd face issue that firewalld won't start after reboot with error Job firewalld.service/start deleted to break ordering cycle starting with network-pre.target/start
Systemd detects this circular dependency at boot and breaks the cycle by:

1. Deleting firewalld.service/start job
2. Allowing network-pre.target to complete immediately (no firewalld to wait for)
3. Network starts → cloud-init completes → sysinit completes → basic completes
4. Boot continues, but firewalld never starts

Steps to reproduce the problem

1. Get instance with Ubuntu 24.04 image that includes cloud-init installed.
2. Install firewalld "apt install firewalld && systemctl enable --now firewalld
3. Restart system/server and get firewalld down

Environment details

• Cloud-init version: /usr/bin/cloud-init 25.2-0ubuntu1~24.04.1
• Operating System Distribution: "24.04.3 LTS (Noble Numbat)" | Linux 6.8.0-85-generic fix invalid json in tools/.lp-to-git-user #85-Ubuntu SMP PREEMPT_DYNAMIC Thu Sep 18 15:26:59 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
• Cloud provider, platform or installer type: DigitalOcean

cloud-init logs

Oct 06 10:49:38 whatchamacallit systemd[1]: Detected virtualization kvm.
Oct 06 10:49:38 whatchamacallit systemd[1]: Detected architecture x86-64.
Oct 06 10:49:38 whatchamacallit systemd[1]: Hostname set to <cooeemobile>.
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found ordering cycle on firewalld.service/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found dependency on polkit.service/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found dependency on basic.target/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found dependency on sockets.target/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found dependency on snapd.socket/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found dependency on sysinit.target/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found dependency on cloud-init.service/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found dependency on systemd-networkd-wait-online.service/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found dependency on systemd-networkd.service/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Found dependency on network-pre.target/start
Oct 06 10:49:38 whatchamacallit systemd[1]: network-pre.target: Job firewalld.service/start deleted to break ordering cycle starting with network-pre.target/start
Oct 06 10:49:38 whatchamacallit systemd[1]: Queued start job for default target graphical.target.

Dependency chain that creates the loop:

  sysinit.target
        ↓ (After=cloud-init.service)
        ↓ [sysinit waits for cloud-init to complete]
        ↓
  cloud-init.service
        ↓ (After=systemd-networkd-wait-online.service)
        ↓ [cloud-init waits for network to be online]
        ↓
  systemd-networkd-wait-online.service
        ↓ (After=systemd-networkd.service)
        ↓ [wait-online waits for networkd to start]
        ↓
  systemd-networkd.service
        ↓ (After=network-pre.target)
        ↓ [networkd waits for network-pre to complete]
        ↓
  network-pre.target
        ↓ (After=firewalld.service)
        ↓ [network-pre waits for firewalld to start]
        ↓
  firewalld.service
        ↓ (After=polkit.service)
        ↓ [firewalld waits for polkit to be ready]
        ↓
  polkit.service
        ↓ (After=basic.target)
        ↓ [polkit waits for basic to complete]
        ↓
  basic.target
        ↓ (Requires=sysinit.target)
        ↓ [basic REQUIRES sysinit to complete first]
        ↓
  sysinit.target => CYCLE!

The dependency does not occur if:

• no firewalld but cloud-init installed
• no cloud-init but firewalld installed

[holmanb]

The linked bug in firewalld was opened back in 2018, so this is pretty old.

Is firewalld actually used on Ubuntu anywhere? ufw is default.

The issue appears to be that firewalld wants to run before network-pre.target but doesn't want to have DefaultDependencies=no.

[uniuuu]

docker with ufw harder to get work due to iptables are managed by docker itself causing overriding.
But in case of firewalld docker can be switched to manage firewalld instead of iptables (iptables completely flushed) and setting strict forward ports allows to get it into control. https://firewalld.org/2024/11/strict-forward-ports

[MajorDallas]

Is firewalld actually used on Ubuntu anywhere? ufw is default.

Ubuntu isn't the only distro that cloud-init might be asked to run on. And, though admittedly rare, some Ubuntu systems do install firewalld--Cisco's CML2 comes to mind.

[holmanb]

Is firewalld actually used on Ubuntu anywhere? ufw is default.

Ubuntu isn't the only distro that cloud-init might be asked to run on.

Indeed, however different distros have opted for a different systemd order.

And, though admittedly rare, some Ubuntu systems do install firewalld--Cisco's CML2 comes to mind.

Does this platform also use cloud-init?

[MajorDallas]

They released a set of terraform modules that render a cloud-config for deployment on various cloud providers: https://github.com/CiscoDevNet/cloud-cml