nvidia hw decoding broken - decoded video is mostly greenish

[Cyborgscode]

System Details:

OS: Fedora 41
Version: <= DAILY BUILD OpenShot-v3.4.0-release-candidate-14702-9a262882-0b018e34-x86_64

Describe the bug:

When nvidia dec or va-api is selected, 3.40 is crashing when loading a mp4 file. (any mp4 file). versions < 3.4 do not crash, they load the video, but the decoding is mostly greenish with some artefacts.
when hw decoding is disabled, openshot works fine.

The attached logs contain all combinations of hw decoding options.

it starts with enabled nvdec decoding.

Log Files:

openshot.tar.gz

Exception / Stacktrace:

none are created

[Cyborgscode]

I can tell, that the breakage happend between 22.08.2025 and 3.10.2025 .

$ vainfo
Trying display: wayland
Trying display: x11
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib64/dri-nonfree/nvidia_drv_video.so
libva info: Trying to open /usr/lib64/dri-freeworld/nvidia_drv_video.so
libva info: Trying to open /usr/lib64/dri/nvidia_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.22 (libva 2.22.0)
vainfo: Driver version: VA-API NVDEC driver [direct backend]
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileH264Main : VAEntrypointVLD
VAProfileH264High : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointVLD
VAProfileVP8Version0_3 : VAEntrypointVLD
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileAV1Profile0 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain12 : VAEntrypointVLD
VAProfileVP9Profile2 : VAEntrypointVLD
VAProfileHEVCMain444 : VAEntrypointVLD
VAProfileHEVCMain444_10 : VAEntrypointVLD
VAProfileHEVCMain444_12 : VAEntrypointVLD

nvidia-smi

Tue Oct 7 21:05:55 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.82.09 Driver Version: 580.82.09 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4060 Off | 00000000:04:00.0 On | N/A |
| 30% 48C P8 N/A / 115W | 779MiB / 8188MiB | 27% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

rpm -qi akmod_nvidia

Name : akmod-nvidia
Epoch : 3
Version : 580.82.09
Release : 1.fc41
Architecture: x86_64
Install Date: Fr 19 Sep 2025 10:01:49 CEST
Group : Unspecified
Size : 98833
License : Redistributable, no modification permitted
Signature : RSA/SHA256, Do 11 Sep 2025 10:20:54 CEST, Key ID 6a2af96194843c65
Source RPM : nvidia-kmod-580.82.09-1.fc41.src.rpm
Build Date : Do 11 Sep 2025 01:29:13 CEST
Build Host : buildvm-07.virt.rpmfusion.net
Packager : RPM Fusion
Vendor : RPM Fusion
URL : https://www.nvidia.com/
Bug URL : https://bugz.rpmfusion.org/nvidia-kmod
Summary : Akmod package for nvidia kernel module(s)
Description :

[Cyborgscode]

BTW: nvidia hw decoding mp4 files with the os-version of ffmpeg works :

fmpeg -hwaccel cuvid -threads 11 -c:v h264_cuvid -hwaccel_output_format cuda -i "$filename" -map 0:1 -map 0:0 -c:v:0 h264_nvenc -b:v:0 $BITS -c:a:0 copy -f mp4 new.mp4

[Colorjet3]

Hardware acceleration in OpenSHot is currently broken and has been for some time. Not sure when the lead developer will get to it.

[Cyborgscode]

hmm.. strange, because it worked at least until the 22th of August, and version 3.3 was in use for months before this.

In mid september, nvidia drivers got updated. I can't downgrade the old driver version atm, so can't check it. But I'm certain, that it's a minor issue to fix, something like the wrong decoder matrix type i.e. RGBA <-> ARGB, something like that kind of mistake.

[Colorjet3]

I don't run Linux so I don't have experience with that. In Windows environment it hasn't worked since v2.6.x.

By the way, are you running the AppImage? If not, see if the AppImage works. You will need to install libfuse2 for the AppImage to start.

[Bert-H]

I wonder if this could be related to issue #5800?
The time-frame when the breakage happened matches up.
Do you run the AppImage? If so, then I'm surprised it's working at all. The AppImage on Fedora42 has been broken since early July. Ever since Fedora updated those MESA drivers.
It's still broken now with latest daily build (14712), MESA drivers at 25.1.9.

[Cyborgscode]

Fedora 41 runs currently mesa 25.0.7
I upgraded to mesa 25.1.9 from fedora 42

Result: runs fine with mesa 25.1.9 and rest of fedora 41. If there is a bug with fedora libs, it's not mesa.

and the reason for it is ofcourse: MESA is NOT LOADED at all, if hwdecoding is disabled.

while testing, if hwdec works now ( stettings window ), it loaded these files:

[pid 10123] openat(AT_FDCWD, "/usr/share/glvnd/egl_vendor.d/50_mesa.json", O_RDONLY) = 182
[pid 10123] openat(AT_FDCWD, "/lib64/libEGL_mesa.so.0", O_RDONLY|O_CLOEXEC) = 182

$ rpm -qf /lib64/libEGL_mesa.so.0
mesa-libEGL-25.1.9-1.fc42.x86_64

hw tests are still failing.

---

I noticed this unrelated error message in the stderr output:

INFO main_window: updateStatusChanged
[8088:8161:1013/090841.507395:ERROR:simple_version_upgrade.cc(165)] Disk cache backend fake index file has wrong magic number.
[8088:8161:1013/090841.507494:ERROR:simple_backend_impl.cc(757)] Simple Cache Backend: wrong file structure on disk: 3 path: /home/xxx/.cache/openshot/QtWebEngine/Default/Cache
INFO timeline: Dropping item on timeline - item_id: OSE0030S3Q, item_type: clip
INFO main_window: updateStatusChanged

maybe a hint for another bug?