🧑‍🌾 ros2node.ros2node.test failing in linux CI

[Crola1702]

Generated by Generative AI

No response

Operating System:

Ubuntu Noble, RHEL 9

ROS version or commit hash:

rolling

RMW implementation (if applicable):

No response

RMW Configuration (if applicable):

No response

Client library (if applicable):

No response

'ros2 doctor --report' output

Steps to reproduce issue

1. Run a build in a linux CI distribution
2. See test regression

Expected behavior

Test should not fail

Actual behavior

Test is failing

Additional information

Reference builds:

• https://ci.ros2.org/view/nightly/job/nightly_linux_release/3473/
• https://ci.ros2.org/view/nightly/job/nightly_linux_debug/3425/
• https://ci.ros2.org/view/nightly/job/nightly_linux-rhel_release/2271/
• https://ci.ros2.org/view/nightly/job/nightly_linux-aarch64_release/3104/
• https://ci.ros2.org/job/nightly_linux_coverage/2773/

Test regression:

• ros2node.ros2node.test.test_cli.test_cli

Log output (similar for all jobs):

===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_list_all_nodes[rmw_fastrtps_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins-agent/workspace/nightly_linux_release/ws/install/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkins-agent/workspace/nightly_linux_release/ws/src/ros2/ros2cli/ros2node/test/test_cli.py", line 135, in test_list_all_nodes
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7e3ee59918a0>(expected_lines=['/_hidden_complex_node', '/complex_node'], text='/_hidden_complex_node\n/cli_bw_delay_hz/cli_bw_delay_hz_test_node\n/complex_node\n/node_ns/test_generate_policy_services_node\n/test_generate_policy_topics_node\n', strict=True)
 +    where <function expect_output at 0x7e3ee59918a0> = <module 'launch_testing.tools' from '/home/jenkins-agent/workspace/nightly_linux_release/ws/install/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/home/jenkins-agent/workspace/nightly_linux_release/ws/install/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   '/_hidden_complex_node\n/cli_bw_delay_hz/cli_bw_delay_hz_test_node\n/complex_node\n/node_ns/test_generate_policy_services_node\n/test_generate_policy_topics_node\n' = <launch_testing.tools.process.ProcessProxy object at 0x7e3ed3e9c6e0>.output


===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_list_node_count[rmw_fastrtps_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins-agent/workspace/nightly_linux_release/ws/install/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkins-agent/workspace/nightly_linux_release/ws/src/ros2/ros2cli/ros2node/test/test_cli.py", line 152, in test_list_node_count
    assert int(output_lines[0]) == 1
AssertionError: assert 2 == 1
 +  where 2 = int('2')


===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_list_nodes[rmw_fastrtps_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins-agent/workspace/nightly_linux_release/ws/install/launch_testing/lib/python3.12/site-packages/launch_testing/markers.py", line 61, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkins-agent/workspace/nightly_linux_release/ws/src/ros2/ros2cli/ros2node/test/test_cli.py", line 124, in test_list_nodes
    assert launch_testing.tools.expect_output(
AssertionError: assert False
 +  where False = <function expect_output at 0x7e3ee59918a0>(expected_lines=['/complex_node'], text='/cli_bw_delay_hz/cli_bw_delay_hz_test_node\n/complex_node\n', strict=True)
 +    where <function expect_output at 0x7e3ee59918a0> = <module 'launch_testing.tools' from '/home/jenkins-agent/workspace/nightly_linux_release/ws/install/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'>.expect_output
 +      where <module 'launch_testing.tools' from '/home/jenkins-agent/workspace/nightly_linux_release/ws/install/launch_testing/lib/python3.12/site-packages/launch_testing/tools/__init__.py'> = launch_testing.tools
 +    and   '/cli_bw_delay_hz/cli_bw_delay_hz_test_node\n/complex_node\n' = <launch_testing.tools.process.ProcessProxy object at 0x7e3ed3ee08c0>.output


===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_info_hidden_node_hidden_flag[rmw_cyclonedds_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Exception: Launch stopped before the active tests finished.


===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_info_hidden_node_no_hidden_flag[rmw_cyclonedds_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Exception: Launch stopped before the active tests finished.


===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_info_node[rmw_cyclonedds_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Exception: Launch stopped before the active tests finished.


===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_list_all_nodes[rmw_cyclonedds_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Exception: Launch stopped before the active tests finished.


===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_list_all_nodes_count[rmw_cyclonedds_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Exception: Launch stopped before the active tests finished.


===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_list_node_count[rmw_cyclonedds_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Exception: Launch stopped before the active tests finished.


===============================================================================================================================================================================================================================================================================================================
FAIL: test_cli.TestROS2NodeCLI.test_list_nodes[rmw_cyclonedds_cpp]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Exception: Launch stopped before the active tests finished.

[christophebedard]

I can't reproduce locally when running just the ros2node tests. However, from just looking at the output for test_cli.TestROS2NodeCLI.test_list_all_nodes[rmw_fastrtps_cpp], it's clear that it only expects 2 nodes to be listed (/_hidden_complex_node and /complex_node) but gets 5 nodes: /_hidden_complex_node, /cli_bw_delay_hz/cli_bw_delay_hz_test_node, /complex_node, /node_ns/test_generate_policy_services_node, /test_generate_policy_topics_node. The extra nodes are coming from sros2 and ros2topic tests.

And it's a similar situation for test_cli.TestROS2NodeCLI.test_list_node_count[rmw_fastrtps_cpp] and test_cli.TestROS2NodeCLI.test_list_nodes[rmw_fastrtps_cpp].

If I run ros2node and ros2topic tests with --executor parallel, then I get test failures with pretty much the same failure output. And looking at the output for test_cli.TestROS2NodeCLI tests with rmw_cyclonedds_cpp, I see the same error as in ros2/rmw_cyclonedds#535:

Starting test run test_cli[rmw_cyclonedds_cpp]
[INFO] [launch]: All log files can be found below /home/christophe.bedard/.ros/log/2025-05-15-14-19-31-830388-ul-pa-1255-1-1398916
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [daemon-stop-28]: process started with pid [1400157]
[INFO] [daemon-stop-28]: process has finished cleanly [pid 1400157]
[INFO] [daemon-start-29]: process started with pid [1400188]
[INFO] [daemon-start-29]: process has finished cleanly [pid 1400188]
[INFO] [python3-30]: process started with pid [1400253]
[INFO] [python3-31]: process started with pid [1400255]
[INFO] [ros2node-cli-32]: process started with pid [1400257]
[python3-31] 1747333200.853966 [0] dq.builtin: type [INVALID 0000000000000000000000000000]: ddsi_xt_type_init_impl with invalid type object

...

Shouldn't these tests be isolated? This leads me to wonder again if something might've regressed with our (domain) isolation mechanism.

[christophebedard]

Scott mentioned that we should move everything to rmw_test_fixture.