[CPU usage] High CPU usage when `rqt_image_view` subscribes to compressed topics

[zymouse]

recognize

• branch: ros2
• commit: c052be0
• pixel_format: "uyvy2rgb"
• image_width: 1920
• image_height: 1080

Test results

• image_raw

• image_raw/compressed

Possible issues arising

High CPU usage for compressed images

Multiple usb cameras are used in autopilot.
When multiple cameras are used at the same time, the usb_cam driver will use up most of the system CPU resources. Affecting the whole autopilot system

Reason for high compressed image usage: Because image_raw has a large transmission bandwidth (190MB/s), the compressed image usage will also be very high.

suggestion

Subscribe to image_raw: Because of the large transmission bandwidth (>100MB/s), the frequency of subscribing to image_raw is reduced. This affects the autopilot perception

Subscribe to /image_raw/compressed: affects image quality and high CPU usage.

We are using opencv's resize method:
Posted by: 1920x1080 -> 540x960
Subscriber: 540x960 -> 1920x1080
This way to avoid - large transfer bandwidth causing frequency degradation and compression increasing CPU usage - image quality degradation issues

ultimate

Thank you very much, a great driver!

[zymouse]

@flynneva Thanks a lot, found the cause of the problem:
Is image_raw too big, use ros2 topic bw :180MB/s,
It seems to be a problem with ROS2 itself, as these problems occur with large ROS2 transfer bandwidths

[flynneva]

@zymouse there is already another issue related to improving the performance of the image_raw topic: #266. That issue is related to the mjpeg2rgb logic though so it might be slightly different to your issue here.

If you are having issues with large-bandwidth topics, have you looked into enabling zero-copy for your image pipeline? Might be worth looking into. I'm not sure if rqt_image_view has that enabled yet though or not...

[firesurfer]

@zymouse I can btw. confirm that this is an issue. I figured out with cyclone dds that with 4x 720P Webcams the raw image topics try to transfer ~700MB/s - on the loopback interface. At this point the kernel starts dropping packages (on all interfaces) which results into a lot of weird behavior.

But this behavior will also depend on the used DDS distribution. Some will use shared memory communication per default for nodes on the same system, some won't per default, some don't even have this feature.

[zymouse]

@firesurfer
It might be possible to use ros2 containers and put the nodes subscribed to the usb_cam topic and the usb_cam node, into a unified container