Lag when using CompressedVideo and 2 image panels when streaming data

[christian-nils]

Hello,

I am not sure if this is related to my unanswered question (https://github.com/orgs/foxglove/discussions/872). But I have experienced something new.

demo.mp4

When opening a Rosbridge connection to my robot and opening more than 2 Image Panels a lag appears. I am using Foxglove Studio 1.86.1.

The frames are 1080p@30fps. I am running Fedora 39 with the snap version of Studio.

Any pointers on preventing this? I will have 2 cameras on my robot, and I would like to be able to use 2 Image Panels.

Thanks in advance,

/CN

[defunctzombie]

@christian-nils Could you give the .deb a try? We've had reports of users experiencing degraded behavior on the snap version and I want to rule that out for your use case.

[christian-nils]

Sure thing, on Wednesday I'll try that. It will be on another computer though since I cannot use .deb packages on Fedora. I hope that is fine.

[christian-nils]

Hello @defunctzombie,

I have now tried with another computer running Ubuntu.
Here is the result with the snap version:

snap-version.mp4

Here is the result with the deb version

deb-version.mp4

Two things to note:

1. The rendering on the image panel for the snap version is worse on this computer. Maybe this is linked to the hardware of the computer. The custom h264 panel has a bit of lag as well but not as much as the official image panel.
2. Comparing both version, I don't see an improvement of the deb version over the snap version.

The deb version was 1.86.1 the snap version was from the release 90 (I forgot to check the actual Foxglove studio version)

I hope that helps

[snosenzo]

@christian-nils
Can you also check whether video decoder hardware acceleration is enabled in chrome/chromium?

You can do that by going here and looking at the results in the console.
https://jsfiddle.net/vmxgb13y/
If hardware h264 support does not succeed that likely means that hardware acceleration is not enabled for chrome (and by-extension probably electron).
To make sure it's enabled for chrome/chromium you can check here: chrome://gpu and ensure that Video Decoding is Hardware accelerated.

Also check that further down the page this table exists and has h264 entries:

If the table does not exist or does not have those entries, that likely means hardware acceleration is disabled for h264 decoding.

If any of the above steps point to hardware acceleration being disabled, you'll probably have to take steps to update or install compatible graphics drivers that would support this (h264 hardware accelerated video decoding). There is not a guarantee that these exist for your specific platform.

You might find some helpful resources here:
https://wiki.archlinux.org/title/Hardware_video_acceleration
https://wiki.archlinux.org/title/Chromium#Hardware_video_acceleration

Hope this helps!

If you could let us know what ended up being the issue here and what steps you took to achieve a solution that would be helpful to us for building up documentation on this issue.

[christian-nils]

Hi @snosenzo !

When running your jsfiddle, I get the following:

"general h264 support", {
  configSucceeded: true,
  configSupport: {
    config: { ... },
    supported: true
  },
  errors: []
}
"hardware h264 support", {
  configSucceeded: false,
  configSupport: {
    config: { ... },
    supported: false
  },
  errors: ["Unsupported configuration. Check isConfigSupported() prior to calling configure()."]
}

Which is in line with what I see on chrome://gpu, i.e., Video Decode has "Hardware accelerated" but nothing is listed on the table you mentioned.

When playing h264 videos online with chrome, the media inspector shows that I am using FFmpeg decoder (without hardware acceleration most probably). So I guess there is an issue of configuration, I did try all what I could following the guide you linked (from ArchLinux wiki) but I cannot get it working on chrome nor chromium.

I am able to use hardware h264 decoding with ffmpeg CLI using CUDA.

Anyway, it seems that I did not find the solution here. The only thing I see is that the custom h264 panel for foxglove studio works without much delays while the image panel does not. I guess the custom panel uses software decoding also.

Have a great day,

[christian-nils]

I have now tried with a Windows 10 computer. I have installed the latest version of Foxglove studio and the experience is worse. I have the h264 hardware decoding listed when checking chrome://gpu:
about-gpu-2024-02-16T13-01-17-872Z.txt

win-version.mp4

I wouldn't blame Windows on the poor results since the hardware of this machine may be a bit weak.

Processor	11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz   2.61 GHz
Installed RAM	16,0 GB (15,7 GB usable)
System type	64-bit operating system, x64-based processor

And the GPU seems to be a Intel Iris Xe Graphics (8GB)

Both panels are extremely delayed compared to reality. But we can see some relative delay between the official Image panel and the custom one (I keep comparing both although I know that the comparison may be unfair as the official panel is much more complex).

[snosenzo]

@christian-nils this past week we pushed a fix that should help the performance of compressed video streams in the image panel if you want to try it out and let us know if it works any better for you. It's only available in the web version so far though.

[christian-nils]

Hi,

Thanks, sure. When you say the web version, you mean studio.foxglove.dev? Or something else? Thanks!

[snosenzo]

Yep, that's it! It should be on the desktop (separate app) version sometime soon though.

[christian-nils]

Hi Sam!

I just tried studio.foxglove.dev, and it works :) a little lag, but absolutely not the same as before so I would like to call this issue fixed.

Thanks!