Skip to content

tests: DNS over IPv6 #696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

tests: DNS over IPv6 #696

wants to merge 2 commits into from
+63 −33

Conversation

marmarek
Copy link
Member

@marmarek marmarek commented Jul 7, 2025

Test if IPv6 DNS works, both if it's the only DNS and in dual-stack setup.

QubesOS/qubes-issues#10038

marmarek added 2 commits July 6, 2025 22:52
@marmarek
tests: extract and deduplicate run_netvm_cmd function
0affed4
@marmarek
tests: DNS over IPv6
d510b59 
Test if IPv6 DNS works, both if it's the only DNS and in dual-stack
setup.

QubesOS/qubes-issues#10038
@codecov
Copy link

codecov bot commented Jul 7, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.83%. Comparing base (929dce2) to head (d510b59).

Additional details and impacted files 
@@           Coverage Diff           @@
##             main     #696   +/-   ##
=======================================
  Coverage   70.83%   70.83%           
=======================================
  Files          61       61           
  Lines       13463    13463           
=======================================
  Hits         9537     9537           
  Misses       3926     3926
Flag Coverage Δ
unittests 70.83% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This was referenced Jul 7, 2025
Open
Merged
@qubesos-bot
Copy link

qubesos-bot commented Jul 7, 2025 •
edited
Loading

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025090403-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025081011-4.3&flavor=update

  • system_tests_basic_vm_qrexec_gui

    • startup: unnamed test (unknown)
    • startup: Failed (test died)
      # Test died: no candidate needle with tag(s) 'nm-connection-establi...

  • system_tests_network

    • VmNetworking_fedora-42-xfce: test_010_simple_proxyvm (failure)
      AssertionError: 2 != 0 : Ping by IP from AppVM failed

    • VmNetworking_fedora-42-xfce: test_030_firewallvm_firewall (failure)
      AssertionError: 2 != 0 : Ping by name failed (should be allowed now)

    • VmNetworking_fedora-42-xfce: test_202_fake_ip_firewall (failure)
      AssertionError: 2 != 0 : Ping by name should be allowed

    • VmNetworking_fedora-42-xfce: test_204_fake_ip_proxy (failure)
      ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 2 != 0

    • VmNetworking_fedora-42-xfce: test_211_custom_ip_proxy (failure)
      ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 2 != 0

    • VmNetworking_fedora-42-xfce: test_212_custom_ip_firewall (failure)
      AssertionError: 2 != 0 : Ping by name should be allowed

  • system_tests_gui_interactive

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_guivm_gui_interactive

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_network_ipv6

    • VmIPv6Networking_debian-13-xfce: test_530_ipv6_firewallvm_firewall (failure)
      AssertionError: 1 != 0 : TCP (IPv4) connection failed (should be al...

    • VmIPv6Networking_debian-13-xfce: test_560_ipv6_dns_only (failure)
      ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 2 != 0

  • system_tests_dispvm

  • system_tests_kde_gui_interactive

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_guivm_vnc_gui_interactive

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_qwt_win10@hw13

    • windows_install: Failed (test died)
      # Test died: command './install.sh' failed at /usr/lib/os-autoinst/...

  • system_tests_qwt_win10_seamless@hw13

    • windows_install: Failed (test died)
      # Test died: command './install.sh' failed at /usr/lib/os-autoinst/...

  • system_tests_qwt_win11@hw13

    • windows_install: Failed (test died)
      # Test died: command './install.sh' failed at /usr/lib/os-autoinst/...

  • system_tests_gui_interactive_preloaded

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_guivm_gpu_gui_interactive@hw13

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • install_default_upload@hw7

    • install_do_user: unnamed test (unknown)

    • install_do_user: Failed (test died)
      # Test died: no candidate needle with tag(s) 'installer-post-instal...

    • install_do_user: wait_serial (wait serial expected)
      # wait_serial expected: qr/4tG2W-\d+-/...

Failed tests

36 failures

  • system_tests_basic_vm_qrexec_gui

    • startup: unnamed test (unknown)
    • startup: Failed (test died)
      # Test died: no candidate needle with tag(s) 'nm-connection-establi...

  • system_tests_network

    • VmNetworking_fedora-42-xfce: test_010_simple_proxyvm (failure)
      AssertionError: 2 != 0 : Ping by IP from AppVM failed

    • VmNetworking_fedora-42-xfce: test_030_firewallvm_firewall (failure)
      AssertionError: 2 != 0 : Ping by name failed (should be allowed now)

    • VmNetworking_fedora-42-xfce: test_202_fake_ip_firewall (failure)
      AssertionError: 2 != 0 : Ping by name should be allowed

    • VmNetworking_fedora-42-xfce: test_204_fake_ip_proxy (failure)
      ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 2 != 0

    • VmNetworking_fedora-42-xfce: test_211_custom_ip_proxy (failure)
      ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 2 != 0

    • VmNetworking_fedora-42-xfce: test_212_custom_ip_firewall (failure)
      AssertionError: 2 != 0 : Ping by name should be allowed

  • system_tests_extra

    • TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure)
      AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

  • system_tests_gui_interactive

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_guivm_gui_interactive

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_network_ipv6

    • VmIPv6Networking_debian-13-xfce: test_530_ipv6_firewallvm_firewall (failure)
      AssertionError: 1 != 0 : TCP (IPv4) connection failed (should be al...

    • VmIPv6Networking_debian-13-xfce: test_560_ipv6_dns_only (failure)
      ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 2 != 0

  • system_tests_dispvm

  • system_tests_kde_gui_interactive

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_guivm_vnc_gui_interactive

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_qwt_win10@hw13

    • windows_install: Failed (test died)
      # Test died: command './install.sh' failed at /usr/lib/os-autoinst/...

  • system_tests_qwt_win10_seamless@hw13

    • windows_install: Failed (test died)
      # Test died: command './install.sh' failed at /usr/lib/os-autoinst/...

  • system_tests_qwt_win11@hw13

    • windows_install: Failed (test died)
      # Test died: command './install.sh' failed at /usr/lib/os-autoinst/...

  • system_tests_gui_interactive_preloaded

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • system_tests_guivm_gpu_gui_interactive@hw13

    • clipboard_and_web: unnamed test (unknown)
    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

  • install_default_upload@hw7

    • install_do_user: unnamed test (unknown)

    • install_do_user: Failed (test died)
      # Test died: no candidate needle with tag(s) 'installer-post-instal...

    • install_do_user: wait_serial (wait serial expected)
      # wait_serial expected: qr/4tG2W-\d+-/...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/149225#dependencies

73 fixed

Unstable tests

Performance Tests

Performance degradation:

15 performance degradations
  • whonix-gateway-17_exec-data-duplex: 81.57 🔺 ( previous job: 68.38, degradation: 119.28%)
  • whonix-workstation-17_exec-data-simplex: 73.82 🔺 ( previous job: 66.80, degradation: 110.50%)
  • whonix-workstation-17_socket-data-duplex: 163.36 🔺 ( previous job: 147.97, degradation: 110.40%)
  • dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 332986.00 :small_red_triangle: ( previous job: 497426.00, degradation: 66.94%)
  • dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 225403.00 :small_red_triangle: ( previous job: 265260.00, degradation: 84.97%)
  • dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 146811.00 :small_red_triangle: ( previous job: 431512.00, degradation: 34.02%)
  • dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 84737.00 :small_red_triangle: ( previous job: 196254.00, degradation: 43.18%)
  • fedora-42-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 77202.00 :small_red_triangle: ( previous job: 86001.00, degradation: 89.77%)
  • fedora-42-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 1054.00 :small_red_triangle: ( previous job: 1368.00, degradation: 77.05%)
  • fedora-42-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 65353.00 :small_red_triangle: ( previous job: 79539.00, degradation: 82.16%)
  • fedora-42-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 624.00 :small_red_triangle: ( previous job: 1251.00, degradation: 49.88%)
  • fedora-42-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 43336.00 :small_red_triangle: ( previous job: 64217.00, degradation: 67.48%)
  • fedora-42-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 76879.00 :small_red_triangle: ( previous job: 87141.00, degradation: 88.22%)
  • fedora-42-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 2521.00 :small_red_triangle: ( previous job: 4098.00, degradation: 61.52%)
  • fedora-42-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 1882.00 :small_red_triangle: ( previous job: 2384.00, degradation: 78.94%)

Remaining performance tests:

57 tests
  • debian-13-xfce_exec: 8.19 🟢 ( previous job: 8.36, improvement: 97.91%)
  • debian-13-xfce_exec-root: 27.72 🔺 ( previous job: 27.36, degradation: 101.30%)
  • debian-13-xfce_socket: 8.43 🟢 ( previous job: 8.57, improvement: 98.34%)
  • debian-13-xfce_socket-root: 7.89 🟢 ( previous job: 8.26, improvement: 95.58%)
  • debian-13-xfce_exec-data-simplex: 60.86 🟢 ( previous job: 72.43, improvement: 84.02%)
  • debian-13-xfce_exec-data-duplex: 74.44 🟢 ( previous job: 76.65, improvement: 97.12%)
  • debian-13-xfce_exec-data-duplex-root: 78.90 🟢 ( previous job: 91.79, improvement: 85.96%)
  • debian-13-xfce_socket-data-duplex: 135.88 🔺 ( previous job: 133.45, degradation: 101.83%)
  • fedora-42-xfce_exec: 9.15 🔺 ( previous job: 9.06, degradation: 101.00%)
  • fedora-42-xfce_exec-root: 59.01 🔺 ( previous job: 58.19, degradation: 101.42%)
  • fedora-42-xfce_socket: 7.49 🟢 ( previous job: 8.48, improvement: 88.37%)
  • fedora-42-xfce_socket-root: 8.37 🔺 ( previous job: 8.18, degradation: 102.27%)
  • fedora-42-xfce_exec-data-simplex: 77.69 🟢 ( previous job: 78.48, improvement: 98.98%)
  • fedora-42-xfce_exec-data-duplex: 69.05 🔺 ( previous job: 67.92, degradation: 101.67%)
  • fedora-42-xfce_exec-data-duplex-root: 104.99 🔺 ( previous job: 96.36, degradation: 108.95%)
  • fedora-42-xfce_socket-data-duplex: 138.27 🟢 ( previous job: 142.58, improvement: 96.97%)
  • whonix-gateway-17_exec: 7.63 🟢 ( previous job: 8.12, improvement: 94.05%)
  • whonix-gateway-17_exec-root: 39.08 🟢 ( previous job: 41.05, improvement: 95.20%)
  • whonix-gateway-17_socket: 7.48 🟢 ( previous job: 8.52, improvement: 87.70%)
  • whonix-gateway-17_socket-root: 8.17 🔺 ( previous job: 8.12, degradation: 100.59%)
  • whonix-gateway-17_exec-data-simplex: 81.20 🟢 ( previous job: 83.60, improvement: 97.13%)
  • whonix-gateway-17_exec-data-duplex-root: 88.19 🟢 ( previous job: 99.37, improvement: 88.75%)
  • whonix-gateway-17_socket-data-duplex: 163.85 🟢 ( previous job: 167.12, improvement: 98.05%)
  • whonix-workstation-17_exec: 7.87 🔺 ( previous job: 7.57, degradation: 103.97%)
  • whonix-workstation-17_exec-root: 57.97 🔺 ( previous job: 56.76, degradation: 102.13%)
  • whonix-workstation-17_socket: 8.65 🔺 ( previous job: 8.59, degradation: 100.77%)
  • whonix-workstation-17_socket-root: 9.18 🔺 ( previous job: 8.89, degradation: 103.29%)
  • whonix-workstation-17_exec-data-duplex: 73.28 🟢 ( previous job: 74.50, improvement: 98.35%)
  • whonix-workstation-17_exec-data-duplex-root: 96.49 🟢 ( previous job: 102.34, improvement: 94.28%)
  • dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 31305.00 :green_circle: ( previous job: 23940.00, improvement: 130.76%)
  • dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 2228.00 :small_red_triangle: ( previous job: 2446.00, degradation: 91.09%)
  • dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 11661.00 :green_circle: ( previous job: 5874.00, improvement: 198.52%)
  • dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 1524.00 :green_circle: ( previous job: 29.00, improvement: 5255.17%)
  • dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 461927.00 :green_circle: ( previous job: 292489.00, improvement: 157.93%)
  • dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 138148.00 :green_circle: ( previous job: 110817.00, improvement: 124.66%)
  • dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 446772.00 :green_circle: ( previous job: 137802.00, improvement: 324.21%)
  • dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 165777.00 :green_circle: ( previous job: 121719.00, improvement: 136.20%)
  • dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 105073.00 :green_circle: ( previous job: 103932.00, improvement: 101.10%)
  • dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 8145.00 :green_circle: ( previous job: 6356.00, improvement: 128.15%)
  • dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 7082.00 :small_red_triangle: ( previous job: 7695.00, degradation: 92.03%)
  • dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 4890.00 :green_circle: ( previous job: 3925.00, improvement: 124.59%)
  • fedora-42-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 402833.00 :green_circle: ( previous job: 366891.00, improvement: 109.80%)
  • fedora-42-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 195542.00 :green_circle: ( previous job: 140215.00, improvement: 139.46%)
  • fedora-42-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 327680.00 :green_circle: ( previous job: 299764.00, improvement: 109.31%)
  • fedora-42-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 79319.00 :green_circle: ( previous job: 47575.00, improvement: 166.72%)
  • fedora-42-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 4039.00 :green_circle: ( previous job: 3020.00, improvement: 133.74%)
  • fedora-42-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 9106.00 :green_circle: ( previous job: 9042.00, improvement: 100.71%)
  • fedora-42-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 351635.00 :small_red_triangle: ( previous job: 387500.00, degradation: 90.74%)
  • fedora-42-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 234633.00 :green_circle: ( previous job: 136640.00, improvement: 171.72%)
  • fedora-42-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 342224.00 :green_circle: ( previous job: 325139.00, improvement: 105.25%)
  • fedora-42-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 91381.00 :green_circle: ( previous job: 87396.00, improvement: 104.56%)
  • fedora-42-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 3768.00 :green_circle: ( previous job: 3765.00, improvement: 100.08%)
  • fedora-42-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 8582.00 :small_red_triangle: ( previous job: 8992.00, degradation: 95.44%)
  • fedora-42-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 373957.00 :small_red_triangle: ( previous job: 383531.00, degradation: 97.50%)
  • fedora-42-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 147247.00 :small_red_triangle: ( previous job: 157382.00, degradation: 93.56%)
  • fedora-42-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 294295.00 :green_circle: ( previous job: 293225.00, improvement: 100.36%)
  • fedora-42-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8293.00 :small_red_triangle: ( previous job: 8804.00, degradation: 94.20%)

This was referenced Jul 7, 2025
Draft
Draft
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Draft
This was referenced Jul 8, 2025
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

openqa-group-2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@marmarek @qubesos-bot