bento/ubuntu-22.04 vagrant box not coming up with libvirt provider

[marc-bh]

Describe the problem

On a Ubuntu 22.04 Jammy system I am using the vagrant from the repo and MUST use the libvirt provider to avoid the VirtualBox conflicts that occur with my other qemu/kvm virtual machines on that Ubuntu 22.04 host.

When I "vagrant up --provider libvirt" using the bento/ubuntu-22.04 box it seems to start coming up properly until it gets to the following line, at which point it hangs for several minutes and eventually times out:

Waiting for domain to get an IP address...

After the timeout it drops a stack trace (below) saying it timed out.

Other systems, such as generic/ubuntu2204 and the bento/ubuntu-24.04 work fine. Given I need libvirt mode I could not use the Canonical Ubuntu vagrant boxes.

It APPEARS, from a tcpdump trace, that the bento/ubuntu-22.04 fails to ever issue a DHCP/bootp to get an IPv4 address assignment, the other Vagrant boxes do and those proceed to come up shortly thereafter. I used a tcpdump on the appropriate vnet interface to find the cause of this hang shortfall for the bento/ubuntu-22.04 box. It's the only one I tried that fails this way.

Software Version

OS: Ubuntu 22.04 with Ubuntu 6.8.0-45 kernel.
Vagrant package versions: vagrant 2.2.19+dfsg-1ubuntu1 ; vagrant-libvirt 0.7.0-1 ; libvirt-daemon 8.0.0-1ubuntu7.10
bento/ubuntu-22.04 - box 'bento/ubuntu-22.04' version '202309.08.0'

Replication Case

Simple from the problem description:

Vagrantfile client VM description:
config.vm.define "client" do |client|
client.vm.box = "bento/ubuntu-22.04"
client.vm.hostname = "client
client.vm.network "private_network", ip: "192.168.58.30", hostname: true
end

vagrant up client --provider libvirt

Stacktrace

NOTE: The backtrace mentions a 2 second timeout(!) but in reality MANY minutes passed before that trace appeared with the error. Things hung for a LONG time at that "Waiting for domain to get an IP address..." line.

marc@dell5540:~/vagrant$ vagrant up client --provider libvirt
Bringing machine 'client' up with 'libvirt' provider...
==> client: Checking if box 'bento/ubuntu-22.04' version '202309.08.0' is up to date...
==> client: Creating image (snapshot of base box volume).
==> client: Creating domain with the following settings...
==> client: -- Name: vagrant_client
==> client: -- Description: Source: /home/marc/vagrant/Vagrantfile
==> client: -- Domain type: qemu
==> client: -- Cpus: 1
==> client: -- Feature: acpi
==> client: -- Feature: apic
==> client: -- Feature: pae
==> client: -- Clock offset: utc
==> client: -- Memory: 512M
==> client: -- Management MAC:
==> client: -- Loader:
==> client: -- Nvram:
==> client: -- Base box: bento/ubuntu-22.04
==> client: -- Storage pool: default
==> client: -- Image(): /var/lib/libvirt/images/vagrant_client.img, 64G
==> client: -- Disk driver opts: cache='default'
==> client: -- Kernel:
==> client: -- Initrd:
==> client: -- Graphics Type: vnc
==> client: -- Graphics Port: -1
==> client: -- Graphics IP: 127.0.0.1
==> client: -- Graphics Password: Not defined
==> client: -- Video Type: cirrus
==> client: -- Video VRAM: 9216
==> client: -- Video 3D accel: false
==> client: -- Sound Type:
==> client: -- Keymap: en-us
==> client: -- TPM Backend: passthrough
==> client: -- TPM Path:
==> client: -- INPUT: type=mouse, bus=ps2
==> client: Creating shared folders metadata...
==> client: Starting domain.
==> client: Waiting for domain to get an IP address...
==> client: Removing domain...
==> client: Deleting the machine folder
/usr/lib/ruby/vendor_ruby/fog/core/wait_for.rb:9:in block in wait_for': The specified wait_for timeout (2 seconds) was exceeded (Fog::Errors::TimeoutError) from /usr/lib/ruby/vendor_ruby/fog/core/wait_for.rb:6:in loop'
from /usr/lib/ruby/vendor_ruby/fog/core/wait_for.rb:6:in wait_for' from /usr/lib/ruby/vendor_ruby/fog/core/model.rb:72:in wait_for'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/driver.rb:204:in get_ipaddress_from_domain' from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/driver.rb:113:in get_domain_ipaddress'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/wait_till_up.rb:45:in block (2 levels) in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/util/retryable.rb:17:in retryable'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/wait_till_up.rb:40:in block in call' from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/util/timer.rb:11:in time'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/wait_till_up.rb:39:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/start_domain.rb:404:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/set_boot_order.rb:80:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/create_network_interfaces.rb:190:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/create_networks.rb:93:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/share_folders.rb:22:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/prepare_nfs_settings.rb:21:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builtin/synced_folders.rb:87:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builtin/delayed.rb:19:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builtin/synced_folder_cleanup.rb:28:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/plugins/synced_folders/nfs/action_cleanup.rb:25:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/prepare_nfs_valid_ids.rb:14:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builtin/provision.rb:80:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/create_domain.rb:443:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/create_domain_volume.rb:94:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/handle_box_image.rb:123:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builtin/handle_box.rb:56:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/handle_storage_pool.rb:63:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/set_name_of_domain.rb:34:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:127:in block in finalize_action' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builder.rb:149:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/runner.rb:89:in block in run'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/util/busy.rb:19:in busy' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/runner.rb:89:in run'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builtin/call.rb:53:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builtin/box_check_outdated.rb:93:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builtin/config_validate.rb:25:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builder.rb:149:in call' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/runner.rb:89:in block in run'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/util/busy.rb:19:in busy' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/runner.rb:89:in run'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/machine.rb:246:in action_raw' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/machine.rb:215:in block in action'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/environment.rb:614:in lock' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/machine.rb:201:in call'
from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/machine.rb:201:in action' from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/batch_action.rb:86:in block (2 levels) in run'

Possible Solution

No idea, just getting started with vagrant. But it would seem there was SOME packaging difference for this box or the kernel within in such that it would not do DHCP/bootp IPv4 address leasing. If there was some workaround to force DHCP or static assignment to the "extra" interface that gets added to each of my VMs then that might be a useful workaround.

[nsballmann]

One workaround is to put config.vm.boot_timeout = 300 into the Vagrantfile. Another option is to remove snapd. In my experience, this issue also exists on non-bento Vagrant boxes with the libvirt provider.