Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AlmaLinux: Issue with H.264 when client request color depth 32 bit (xorgxrdp) #3405

Open
hauihau opened this issue Jan 22, 2025 · 24 comments
Open

Comments

@hauihau
Copy link

hauihau commented Jan 22, 2025

xrdp version

0.10.2

Detailed xrdp version, build options

xrdp 0.10.2
  A Remote Desktop Protocol Server.
  Copyright (C) 2004-2024 Jay Sorg, Neutrino Labs, and all contributors.
  See https://github.com/neutrinolabs/xrdp for more information.

  Configure options:
      --build=x86_64-redhat-linux-gnu
      --host=x86_64-redhat-linux-gnu
      --program-prefix=
      --disable-dependency-tracking
      --prefix=/usr
      --exec-prefix=/usr
      --bindir=/usr/bin
      --sbindir=/usr/sbin
      --sysconfdir=/etc
      --datadir=/usr/share
      --includedir=/usr/include
      --libdir=/usr/lib64
      --libexecdir=/usr/libexec
      --localstatedir=/var
      --sharedstatedir=/var/lib
      --mandir=/usr/share/man
      --infodir=/usr/share/info
      --enable-fuse
      --enable-pixman
      --enable-painter
      --enable-vsock
      --enable-ipv6
      --enable-openh264
      --with-socketdir=/run/xrdp
      --with-imlib2
      build_alias=x86_64-redhat-linux-gnu
      host_alias=x86_64-redhat-linux-gnu
      CC=gcc
      CFLAGS=-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
      LDFLAGS=-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
      CXX=g++
      CXXFLAGS=-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
      LT_SYS_LIBRARY_PATH=/usr/lib64:
      PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig

  Compiled with OpenSSL 3.0.7 1 Nov 2022

Operating system & version

RedHat Enterprise Linix 9.4 / Almalinux 9.5

Installation method

dnf / apt / zypper / pkg / etc

Which backend do you use?

xorgxrdp 0.10.3

What desktop environment do you use?

Xfce

Environment xrdp running on

Physical and virtual systems

What's your client?

Windows 11 Microsoft Client

Area(s) with issue?

Graphic glitches, Other

Steps to reproduce

We have just updates to the lastest version of xrdp and xorgxrdp on our RedHat and Alma Linux systems.

When we connect to the systems with 32 bit color depth, we only see a blue screen and the systemd journal contains lot's of lines with

[ERROR] Failed to create H.264 encoder

If we reduce the color depth to 24 or less, everything is working as expected.

There is also in the logs when conencting:

Color depth 32 bit:

[INFO ] client supports gfx protocol

Color depth <32 bit:

[WARN ] client requested gfx protocol with insufficient color depth

✔️ Expected Behavior

Seeing the Xfce desktop

❌ Actual Behavior

The window has just blue background, no Xfce desktop items visible

Anything else?

Please let me know if you need further details or logs.

@hauihau hauihau added the bug label Jan 22, 2025
@matt335672
Copy link
Member

@hauihau - did you install xrdp using dnf? It isn't clear from your description above.

@matt335672
Copy link
Member

I've had a poke around in the source, and this looks currently like a memory allocation error of some sort.

Are you able to provide the following?

  • An mstsc.exe ini file which exhibits this?
  • The layout of the monitor or monitors you are using with xrdp?

Thanks.

@hauihau
Copy link
Author

hauihau commented Jan 23, 2025

@hauihau - did you install xrdp using dnf? It isn't clear from your description above.

Both xrdp and xorgxrdp were installed with dnf from EPEL 9 repository,

@hauihau
Copy link
Author

hauihau commented Jan 23, 2025

I've had a poke around in the source, and this looks currently like a memory allocation error of some sort.

Are you able to provide the following?

  • An mstsc.exe ini file which exhibits this?
  • The layout of the monitor or monitors you are using with xrdp?

Thanks.

I'm not aware of any ini file, bit this is the content of the Default.rdp:

screen mode id:i:2
use multimon:i:0
desktopwidth:i:2560
desktopheight:i:1440
session bpp:i:24
winposstr:s:0,1,449,203,1249,803
compression:i:1
keyboardhook:i:2
audiocapturemode:i:0
videoplaybackmode:i:1
connection type:i:7
networkautodetect:i:1
bandwidthautodetect:i:1
displayconnectionbar:i:1
enableworkspacereconnect:i:0
disable wallpaper:i:0
allow font smoothing:i:0
allow desktop composition:i:0
disable full window drag:i:1
disable menu anims:i:1
disable themes:i:0
disable cursor setting:i:0
bitmapcachepersistenable:i:1
full address:s:
audiomode:i:0
redirectprinters:i:1
redirectlocation:i:0
redirectcomports:i:0
redirectsmartcards:i:1
redirectwebauthn:i:1
redirectclipboard:i:1
redirectposdevices:i:0
autoreconnection enabled:i:1
authentication level:i:2
prompt for credentials:i:0
negotiate security layer:i:1
remoteapplicationmode:i:0
alternate shell:s:
shell working directory:s:
gatewayhostname:s:
gatewayusagemethod:i:4
gatewaycredentialssource:i:4
gatewayprofileusagemethod:i:0
promptcredentialonce:i:0
gatewaybrokeringtype:i:0
use redirection server name:i:0
rdgiskdcproxy:i:0
kdcproxyname:s:
enablerdsaadauth:i:0
drivestoredirect:s:
username:s:

I have two monitors connected to a USB-C/Thunderbolt docking station, both displays have 2560x1440.

@matt335672
Copy link
Member

Thanks @hauihau - I'll try to reproduce this here using similar settings.

@matt335672
Copy link
Member

I'm having a few problems with the debuffer on Alma 9, so I've not managed to reproduce this yet.

A thought occurs to me however; what do you get for the following command?

rpm -qa | grep -i h264

@hauihau
Copy link
Author

hauihau commented Jan 24, 2025

I'm having a few problems with the debuffer on Alma 9, so I've not managed to reproduce this yet.

A thought occurs to me however; what do you get for the following command?

rpm -qa | grep -i h264

noopenh264-0.1.0~openh264_2.4.1-2.el9.x86_64

@thomaspmorgan
Copy link

I have the same issue on RHEL 9.5 with latest updates. Was working before latest update.

@matt335672
Copy link
Member

I think I can see what's going on here.

With noopenh264 installed:-

$ rpm -qi noopenh264-0.1.0~openh264_2.4.1-2.el9.x86_64
Name        : noopenh264
. . .
Summary     : Fake implementation of the OpenH264 library
Description :
Fake implementation of the OpenH264 library we can link from
regardless of the actual library being available.

You should both have openh264 installed:-

$ rpm -qi openh264-2.3.1-1.el9.x86_64
Name        : openh264
. . .

Summary     : H.264 codec library
Description :
OpenH264 is a codec library which supports H.264 encoding and decoding. It is
suitable for use in real time applications such as WebRTC.

My (possibly incorrect) understanding is that there are some concerns over H264 patents which is why the noopenh264 package exists.

I'll need to look into why this has happened with a fresh Alma install.

The following command might fix things, but I'd only try it on a test system for the time being until I've looked in more detail. Please don't run this in production yet!

sudo dnf install openh264 --allowerasing

@matt335672
Copy link
Member

I can see where noopenh264 could come from. It's a bit complicated, and the explanation is correspondingly long:-

I've made a vanilla installation of Alma 9.5 with EPEL using this procedure:-

  1. Install base system from iso
  2. sudo dnf update
  3. sudo dnf install epel-release
  4. sudo dnf groupinstall Xfce base-x
  5. sudo systemctl set-default graphical
  6. sudo reboot

Then:-

$ sudo dnf install xrdp
Last metadata expiration check: 1:55:56 ago on Sat Jan 25 14:36:39 2025.
Dependencies resolved.
================================================================================
 Package                  Arch    Version                      Repository  Size
================================================================================
Installing:
 xrdp                     x86_64  1:0.10.2-7.el9               epel       593 k
Installing dependencies:
 dbus-x11                 x86_64  1:1.12.20-8.el9              appstream   24 k
 fuse3-libs               x86_64  3.10.2-9.el9                 appstream   90 k
 imlib2                   x86_64  1.7.4-1.el9                  epel       225 k
 noopenh264               x86_64  0.1.0~openh264_2.4.1-2.el9   epel        22 k
 tigervnc-license         noarch  1.14.1-1.el9_5               appstream   17 k
 tigervnc-server-minimal  x86_64  1.14.1-1.el9_5               appstream  1.2 M
Installing weak dependencies:
 xrdp-selinux             x86_64  1:0.10.2-7.el9               epel        13 k

Transaction Summary
================================================================================
Install  8 Packages

Total download size: 2.1 M
Installed size: 7.4 M
Is this ok [y/N]: 

There is no sign of the openh264 package which is actually needed by xrdp:-

$ sudo dnf search openh264
Last metadata expiration check: 0:00:07 ago on Sat Jan 25 16:52:04 2025.
======================= Name & Summary Matched: openh264 =======================
noopenh264.x86_64 : Fake implementation of the OpenH264 library
noopenh264-devel.x86_64 : Development files for noopenh264

$ sudo dnf repolist --all
repo id                    repo name                                    status
appstream                  AlmaLinux 9 - AppStream                      enabled
appstream-debuginfo        AlmaLinux 9 - AppStream - Debug              disabled
appstream-source           AlmaLinux 9 - AppStream - Source             disabled
baseos                     AlmaLinux 9 - BaseOS                         enabled
baseos-debuginfo           AlmaLinux 9 - BaseOS - Debug                 disabled
baseos-source              AlmaLinux 9 - BaseOS - Source                disabled
crb                        AlmaLinux 9 - CRB                            disabled
crb-debuginfo              AlmaLinux 9 - CRB - Debug                    disabled
crb-source                 AlmaLinux 9 - CRB - Source                   disabled
epel                       Extra Packages for Enterprise Linux 9 - x86_ enabled
epel-debuginfo             Extra Packages for Enterprise Linux 9 - x86_ disabled
epel-source                Extra Packages for Enterprise Linux 9 - x86_ disabled
epel-testing               Extra Packages for Enterprise Linux 9 - Test disabled
epel-testing-debuginfo     Extra Packages for Enterprise Linux 9 - Test disabled
epel-testing-source        Extra Packages for Enterprise Linux 9 - Test disabled
extras                     AlmaLinux 9 - Extras                         enabled
extras-debuginfo           AlmaLinux 9 - Extras - Debug                 disabled
extras-source              AlmaLinux 9 - Extras - Source                disabled
highavailability           AlmaLinux 9 - HighAvailability               disabled
highavailability-debuginfo AlmaLinux 9 - HighAvailability - Debug       disabled
highavailability-source    AlmaLinux 9 - HighAvailability - Source      disabled
nfv                        AlmaLinux 9 - NFV                            disabled
nfv-debuginfo              AlmaLinux 9 - NFV - Debug                    disabled
nfv-source                 AlmaLinux 9 - NFV - Source                   disabled
plus                       AlmaLinux 9 - Plus                           disabled
plus-debuginfo             AlmaLinux 9 - Plus - Debug                   disabled
plus-source                AlmaLinux 9 - Plus - Source                  disabled
resilientstorage           AlmaLinux 9 - ResilientStorage               disabled
resilientstorage-debuginfo AlmaLinux 9 - ResilientStorage - Debug       disabled
resilientstorage-source    AlmaLinux 9 - ResilientStorage - Source      disabled
rt                         AlmaLinux 9 - RT                             disabled
rt-debuginfo               AlmaLinux 9 - RT - Debug                     disabled
rt-source                  AlmaLinux 9 - RT - Source                    disabled
sap                        AlmaLinux 9 - SAP                            disabled
sap-debuginfo              AlmaLinux 9 - SAP - Debug                    disabled
sap-source                 AlmaLinux 9 - SAP - Source                   disabled
saphana                    AlmaLinux 9 - SAPHANA                        disabled
saphana-debuginfo          AlmaLinux 9 - SAPHANA - Debug                disabled
saphana-source             AlmaLinux 9 - SAPHANA - Source               disabled

However, if at this point, I run a dnf-update, the version of the epel-release package gets updated from 9.5 to 9.9:-

Name        : epel-release
Version     : 9
Release     : 5.el9
Architecture: noarch
Install Date: Sat Jan 25 13:44:18 2025
Group       : Unspecified
Size        : 25723
License     : GPLv2
Signature   : RSA/SHA256, Mon Aug 21 14:35:39 2023, Key ID d36cb86cb86b3716
Source RPM  : epel-release-9-5.el9.src.rpm
Build Date  : Mon Aug 21 13:56:23 2023
Build Host  : s390x-builder01.almalinux.org
Packager    : AlmaLinux Packaging Team <[email protected]>
Vendor      : AlmaLinux
URL         : http://download.fedoraproject.org/pub/epel
Summary     : Extra Packages for Enterprise Linux repository configuration
Description :
This package contains the Extra Packages for Enterprise Linux (EPEL) repository
GPG key as well as configuration for yum.

$ sudo dnf update
Last metadata expiration check: 0:02:50 ago on Sat Jan 25 16:52:04 2025.
Dependencies resolved.
================================================================================
 Package                Architecture     Version           Repository      Size
================================================================================
Upgrading:
 epel-release           noarch           9-9.el9           epel            19 k

Transaction Summary
================================================================================
Upgrade  1 Package

Total download size: 19 k
Is this ok [y/N]: y
Downloading Packages:
epel-release-9-9.el9.noarch.rpm                  94 kB/s |  19 kB     00:00    
--------------------------------------------------------------------------------
Total                                            39 kB/s |  19 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Upgrading        : epel-release-9-9.el9.noarch                            1/2 
  Running scriptlet: epel-release-9-9.el9.noarch                            1/2 
  Cleanup          : epel-release-9-5.el9.noarch                            2/2 
  Running scriptlet: epel-release-9-5.el9.noarch                            2/2 
  Verifying        : epel-release-9-9.el9.noarch                            1/2 
  Verifying        : epel-release-9-5.el9.noarch                            2/2 

Upgraded:
  epel-release-9-9.el9.noarch                                                   

Complete!

$ rpm -qi epel-release
Name        : epel-release
Version     : 9
Release     : 9.el9
Architecture: noarch
Install Date: Sat Jan 25 16:54:57 2025
Group       : Unspecified
Size        : 26551
License     : GPLv2
Signature   : RSA/SHA256, Tue Dec  3 01:04:39 2024, Key ID 8a3872bf3228467c
Source RPM  : epel-release-9-9.el9.src.rpm
Build Date  : Mon Dec  2 23:59:07 2024
Build Host  : buildvm-x86-05.iad2.fedoraproject.org
Packager    : Fedora Project
Vendor      : Fedora Project
URL         : http://download.fedoraproject.org/pub/epel
Bug URL     : https://bugz.fedoraproject.org/epel-release
Summary     : Extra Packages for Enterprise Linux repository configuration
Description :
This package contains the Extra Packages for Enterprise Linux (EPEL) repository
GPG key as well as configuration for yum.

Now I can see the openh264 package:-

$ sudo dnf search openh264
Extra Packages for Enterprise Linux 9 openh264  4.7 kB/s | 2.5 kB     00:00    
Last metadata expiration check: 0:00:01 ago on Sat Jan 25 16:56:06 2025.
======================== Name Exactly Matched: openh264 ========================
openh264.x86_64 : H.264 codec library
======================= Name & Summary Matched: openh264 =======================
noopenh264.x86_64 : Fake implementation of the OpenH264 library
noopenh264-devel.x86_64 : Development files for noopenh264
openh264-devel.x86_64 : Development files for openh264
============================ Name Matched: openh264 ============================
gstreamer1-plugin-openh264.x86_64 : GStreamer H.264 plugin
mozilla-openh264.x86_64 : H.264 codec support for Mozilla browsers

$ sudo dnf repolist --all
repo id                       repo name                                 status
appstream                     AlmaLinux 9 - AppStream                   enabled
appstream-debuginfo           AlmaLinux 9 - AppStream - Debug           disabled
appstream-source              AlmaLinux 9 - AppStream - Source          disabled
baseos                        AlmaLinux 9 - BaseOS                      enabled
baseos-debuginfo              AlmaLinux 9 - BaseOS - Debug              disabled
baseos-source                 AlmaLinux 9 - BaseOS - Source             disabled
crb                           AlmaLinux 9 - CRB                         disabled
crb-debuginfo                 AlmaLinux 9 - CRB - Debug                 disabled
crb-source                    AlmaLinux 9 - CRB - Source                disabled
epel                          Extra Packages for Enterprise Linux 9 - x enabled
epel-cisco-openh264           Extra Packages for Enterprise Linux 9 ope enabled
epel-cisco-openh264-debuginfo Extra Packages for Enterprise Linux 9 ope disabled
epel-cisco-openh264-source    Extra Packages for Enterprise Linux 9 ope disabled
epel-debuginfo                Extra Packages for Enterprise Linux 9 - x disabled
epel-source                   Extra Packages for Enterprise Linux 9 - x disabled
epel-testing                  Extra Packages for Enterprise Linux 9 - T disabled
epel-testing-debuginfo        Extra Packages for Enterprise Linux 9 - T disabled
epel-testing-source           Extra Packages for Enterprise Linux 9 - T disabled
extras                        AlmaLinux 9 - Extras                      enabled
extras-debuginfo              AlmaLinux 9 - Extras - Debug              disabled
extras-source                 AlmaLinux 9 - Extras - Source             disabled
highavailability              AlmaLinux 9 - HighAvailability            disabled
highavailability-debuginfo    AlmaLinux 9 - HighAvailability - Debug    disabled
highavailability-source       AlmaLinux 9 - HighAvailability - Source   disabled
nfv                           AlmaLinux 9 - NFV                         disabled
nfv-debuginfo                 AlmaLinux 9 - NFV - Debug                 disabled
nfv-source                    AlmaLinux 9 - NFV - Source                disabled
plus                          AlmaLinux 9 - Plus                        disabled
plus-debuginfo                AlmaLinux 9 - Plus - Debug                disabled
plus-source                   AlmaLinux 9 - Plus - Source               disabled
resilientstorage              AlmaLinux 9 - ResilientStorage            disabled
resilientstorage-debuginfo    AlmaLinux 9 - ResilientStorage - Debug    disabled
resilientstorage-source       AlmaLinux 9 - ResilientStorage - Source   disabled
rt                            AlmaLinux 9 - RT                          disabled
rt-debuginfo                  AlmaLinux 9 - RT - Debug                  disabled
rt-source                     AlmaLinux 9 - RT - Source                 disabled
sap                           AlmaLinux 9 - SAP                         disabled
sap-debuginfo                 AlmaLinux 9 - SAP - Debug                 disabled
sap-source                    AlmaLinux 9 - SAP - Source                disabled
saphana                       AlmaLinux 9 - SAPHANA                     disabled
saphana-debuginfo             AlmaLinux 9 - SAPHANA - Debug             disabled
saphana-source                AlmaLinux 9 - SAPHANA - Source            disabled

and, more importantly:-

$ sudo dnf install xrdp
Last metadata expiration check: 0:01:19 ago on Sat Jan 25 16:56:06 2025.
Dependencies resolved.
================================================================================
 Package                  Arch    Version            Repository            Size
================================================================================
Installing:
 xrdp                     x86_64  1:0.10.2-7.el9     epel                 593 k
Installing dependencies:
 dbus-x11                 x86_64  1:1.12.20-8.el9    appstream             24 k
 fuse3-libs               x86_64  3.10.2-9.el9       appstream             90 k
 imlib2                   x86_64  1.7.4-1.el9        epel                 225 k
 openh264                 x86_64  2.3.1-1.el9        epel-cisco-openh264  425 k
 tigervnc-license         noarch  1.14.1-1.el9_5     appstream             17 k
 tigervnc-server-minimal  x86_64  1.14.1-1.el9_5     appstream            1.2 M
Installing weak dependencies:
 xrdp-selinux             x86_64  1:0.10.2-7.el9     epel                  13 k

Transaction Summary
================================================================================
Install  8 Packages

Total download size: 2.5 M
Installed size: 8.5 M
Is this ok [y/N]: 

Carrying on with the installation works perfectly.

What is happening here, is that the epel-release which comes with AlmaLinux (9.5) does not have the file /etc/yum.repos.d/epel-cisco-openh264.repo in it. There is an update to this package within EPEL itself. To get this, once you have installed the epel-release package, you must update it before installing xrdp.

If you are in the situation where xrdp has been installed along with noopenh264, proceed as follows:-

  1. Check what depends on noopenh264 by trying to remove it:-

    $ sudo dnf --assumeno remove noopenh264
    Dependencies resolved.
    ================================================================================
     Package                  Arch    Version                     Repository   Size
    ================================================================================
    Removing:
     noopenh264               x86_64  0.1.0~openh264_2.4.1-2.el9  @epel        43 k
    Removing dependent packages:
     xorgxrdp                 x86_64  0.10.3-5.el9                @epel       178 k
     xrdp                     x86_64  1:0.10.2-7.el9              @epel       3.4 M
    Removing unused dependencies:
     dbus-x11                 x86_64  1:1.12.20-8.el9             @appstream   36 k
     fuse3-libs               x86_64  3.10.2-9.el9                @appstream  281 k
     imlib2                   x86_64  1.7.4-1.el9                 @epel       744 k
     tigervnc-license         noarch  1.14.1-1.el9_5              @appstream   18 k
     tigervnc-server-minimal  x86_64  1.14.1-1.el9_5              @appstream  2.8 M
     xrdp-selinux             x86_64  1:0.10.2-7.el9              @epel       9.0 k
    
    Transaction Summary
    ================================================================================
    Remove  9 Packages
    
    Freed space: 7.5 M
    Operation aborted.
    

    The above example shows only xrdp and xorgxrdp are affected. So it's safe to proceed.

  2. Manually kill any active xrdp sessions.

  3. Make sure epel-release is up to date with sudo dnf update epel-release ; sudo dnf update

  4. Replace noopenh264 with openh264 with sudo dnf install openh264 --allowerasing

  5. Restart xrdp with sudo systemctl restart xrdp

@matt335672 matt335672 changed the title Issue with H.264 when client request color depth 32 bit AlmaLinux: Issue with H.264 when client request color depth 32 bit (xorgxrdp) Jan 25, 2025
@metalefty
Copy link
Member

metalefty commented Jan 27, 2025

I doubt the issue is AlmaLinux specific because AlmaLinux doesn't have xrdp package. It comes from EPEL. Also openh264/noopenh264 comes from EPEL. However, I'll check it anyway

@matt335672
Copy link
Member

My thinking at the moment is that it is Alma-specific

Alma includes an epel-release package that doesn't contain /etc/yum.repos.d/epel-cisco-openh264.repo.

On both of these scenarios I'm starting with a clean, fully updated Alma installation, without epel-release installed.

Scenario 1 - install epel-release, install xrdp

$ sudo dnf install epel-release
Last metadata expiration check: 0:05:39 ago on Mon Jan 27 09:26:27 2025.
Dependencies resolved.
================================================================================
 Package               Architecture    Version            Repository       Size
================================================================================
Installing:
 epel-release          noarch          9-5.el9            extras           18 k

Transaction Summary
================================================================================
Install  1 Package

Total download size: 18 k
Installed size: 25 k
Is this ok [y/N]: y
Downloading Packages:
epel-release-9-5.el9.noarch.rpm                 154 kB/s |  18 kB     00:00    
--------------------------------------------------------------------------------
Total                                            28 kB/s |  18 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Installing       : epel-release-9-5.el9.noarch                            1/1 
  Running scriptlet: epel-release-9-5.el9.noarch                            1/1 
Many EPEL packages require the CodeReady Builder (CRB) repository.
It is recommended that you run /usr/bin/crb enable to enable the CRB repository.

  Verifying        : epel-release-9-5.el9.noarch                            1/1 

Installed:
  epel-release-9-5.el9.noarch                                                   

Complete!
$ sudo dnf install xrdp
Last metadata expiration check: 0:05:46 ago on Mon Jan 27 09:26:30 2025.
Dependencies resolved.
================================================================================
 Package                  Arch    Version                      Repository  Size
================================================================================
Installing:
 xrdp                     x86_64  1:0.10.2-7.el9               epel       593 k
Installing dependencies:
 dbus-x11                 x86_64  1:1.12.20-8.el9              appstream   24 k
 fuse3-libs               x86_64  3.10.2-9.el9                 appstream   90 k
 imlib2                   x86_64  1.7.4-1.el9                  epel       225 k
 noopenh264               x86_64  0.1.0~openh264_2.4.1-2.el9   epel        22 k
 tigervnc-license         noarch  1.14.1-1.el9_5               appstream   17 k
 tigervnc-server-minimal  x86_64  1.14.1-1.el9_5               appstream  1.2 M
Installing weak dependencies:
 xrdp-selinux             x86_64  1:0.10.2-7.el9               epel        13 k

Transaction Summary
================================================================================
Install  8 Packages

Total download size: 2.1 M
Installed size: 7.4 M
Is this ok [y/N]: n
Operation aborted.

The noopenh264 package is installed, resulting in the symptoms in this issue

Scenario 2 - install epel-release, update, install xrdp

$ sudo dnf install epel-release
Last metadata expiration check: 0:06:51 ago on Mon Jan 27 09:26:27 2025.
Dependencies resolved.
================================================================================
 Package               Architecture    Version            Repository       Size
================================================================================
Installing:
 epel-release          noarch          9-5.el9            extras           18 k

Transaction Summary
================================================================================
Install  1 Package

Total download size: 18 k
Installed size: 25 k
Is this ok [y/N]: y
Downloading Packages:
epel-release-9-5.el9.noarch.rpm                 250 kB/s |  18 kB     00:00    
--------------------------------------------------------------------------------
Total                                            41 kB/s |  18 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Installing       : epel-release-9-5.el9.noarch                            1/1 
  Running scriptlet: epel-release-9-5.el9.noarch                            1/1 
Many EPEL packages require the CodeReady Builder (CRB) repository.
It is recommended that you run /usr/bin/crb enable to enable the CRB repository.

  Verifying        : epel-release-9-5.el9.noarch                            1/1 

Installed:
  epel-release-9-5.el9.noarch                                                   

Complete!
$ sudo dnf update
Last metadata expiration check: 0:07:02 ago on Mon Jan 27 09:26:30 2025.
Dependencies resolved.
================================================================================
 Package                Architecture     Version           Repository      Size
================================================================================
Upgrading:
 epel-release           noarch           9-9.el9           epel            19 k

Transaction Summary
================================================================================
Upgrade  1 Package

Total download size: 19 k
Is this ok [y/N]: y
Downloading Packages:
epel-release-9-9.el9.noarch.rpm                  52 kB/s |  19 kB     00:00    
--------------------------------------------------------------------------------
Total                                            21 kB/s |  19 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Upgrading        : epel-release-9-9.el9.noarch                            1/2 
  Running scriptlet: epel-release-9-9.el9.noarch                            1/2 
  Cleanup          : epel-release-9-5.el9.noarch                            2/2 
  Running scriptlet: epel-release-9-5.el9.noarch                            2/2 
  Verifying        : epel-release-9-9.el9.noarch                            1/2 
  Verifying        : epel-release-9-5.el9.noarch                            2/2 

Upgraded:
  epel-release-9-9.el9.noarch                                                   

Complete!
$ sudo dnf install xrdp
Extra Packages for Enterprise Linux 9 openh264  4.6 kB/s | 2.5 kB     00:00    
Dependencies resolved.
================================================================================
 Package                  Arch    Version            Repository            Size
================================================================================
Installing:
 xrdp                     x86_64  1:0.10.2-7.el9     epel                 593 k
Installing dependencies:
 dbus-x11                 x86_64  1:1.12.20-8.el9    appstream             24 k
 fuse3-libs               x86_64  3.10.2-9.el9       appstream             90 k
 imlib2                   x86_64  1.7.4-1.el9        epel                 225 k
 openh264                 x86_64  2.3.1-1.el9        epel-cisco-openh264  425 k
 tigervnc-license         noarch  1.14.1-1.el9_5     appstream             17 k
 tigervnc-server-minimal  x86_64  1.14.1-1.el9_5     appstream            1.2 M
Installing weak dependencies:
 xrdp-selinux             x86_64  1:0.10.2-7.el9     epel                  13 k

Transaction Summary
================================================================================
Install  8 Packages

Total download size: 2.5 M
Installed size: 8.5 M
Is this ok [y/N]: n
Operation aborted.

The openh264 package is installed, resulting in correct operation.

The corresponding Rocky package does contain the /etc/yum.repos.d/epel-cisco-openh264.repo file, so I suspect it will work OK.

I'm just downloading Rocky, so I can test out my assertion and report back.

@matt335672
Copy link
Member

matt335672 commented Jan 27, 2025

Rocky 9.5 - install epel-release, install xrdp

To speed things up, I've used a minimal install of Rocky (no GUI).

$ sudo dnf install epel-release
Last metadata expiration check: 0:01:06 ago on Mon 27 Jan 2025 09:57:04 GMT.
Dependencies resolved.
================================================================================
 Package               Architecture    Version            Repository       Size
================================================================================
Installing:
 epel-release          noarch          9-7.el9            extras           19 k

Transaction Summary
================================================================================
Install  1 Package

Total download size: 19 k
Installed size: 26 k
Is this ok [y/N]: y
Downloading Packages:
epel-release-9-7.el9.noarch.rpm                 121 kB/s |  19 kB     00:00    
--------------------------------------------------------------------------------
Total                                            57 kB/s |  19 kB     00:00     
Rocky Linux 9 - Extras                          1.7 MB/s | 1.7 kB     00:00    
Importing GPG key 0x350D275D:
 Userid     : "Rocky Enterprise Software Foundation - Release key 2022 <[email protected]>"
 Fingerprint: 21CB 256A E16F C54C 6E65 2949 702D 426D 350D 275D
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-9
Is this ok [y/N]: y
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Installing       : epel-release-9-7.el9.noarch                            1/1 
  Running scriptlet: epel-release-9-7.el9.noarch                            1/1 
Many EPEL packages require the CodeReady Builder (CRB) repository.
It is recommended that you run /usr/bin/crb enable to enable the CRB repository.

  Verifying        : epel-release-9-7.el9.noarch                            1/1 

Installed:
  epel-release-9-7.el9.noarch                                                   

Complete!
$ sudo dnf install xrdp
Extra Packages for Enterprise Linux 9 - x86_64  6.7 MB/s |  23 MB     00:03    
Extra Packages for Enterprise Linux 9 openh264  6.4 kB/s | 2.5 kB     00:00    
Dependencies resolved.
================================================================================
 Package                  Arch    Version            Repository            Size
================================================================================
Installing:
 xrdp                     x86_64  1:0.10.2-7.el9     epel                 593 k
Installing dependencies:
 dbus-daemon              x86_64  1:1.12.20-8.el9    appstream            197 k
 dbus-tools               x86_64  1:1.12.20-8.el9    baseos                50 k
 dbus-x11                 x86_64  1:1.12.20-8.el9    appstream             24 k
 freetype                 x86_64  2.10.4-9.el9       baseos               387 k
 fuse3-libs               x86_64  3.10.2-9.el9       appstream             91 k
 giflib                   x86_64  5.2.1-9.el9        appstream             48 k
 graphite2                x86_64  1.3.14-9.el9       baseos                94 k
 harfbuzz                 x86_64  2.7.4-10.el9       baseos               623 k
 imlib2                   x86_64  1.7.4-1.el9        epel                 225 k
 jbigkit-libs             x86_64  2.1-23.el9         appstream             52 k
 libICE                   x86_64  1.0.10-8.el9       appstream             70 k
 libSM                    x86_64  1.2.3-10.el9       appstream             41 k
 libX11                   x86_64  1.7.0-9.el9        appstream            650 k
 libX11-common            noarch  1.7.0-9.el9        appstream            151 k
 libX11-xcb               x86_64  1.7.0-9.el9        appstream             10 k
 libXau                   x86_64  1.0.9-8.el9        appstream             30 k
 libXcursor               x86_64  1.2.0-7.el9        appstream             30 k
 libXdmcp                 x86_64  1.1.3-8.el9        appstream             35 k
 libXext                  x86_64  1.3.4-8.el9        appstream             39 k
 libXfixes                x86_64  5.0.3-16.el9       appstream             19 k
 libXfont2                x86_64  2.0.3-12.el9       appstream            143 k
 libXi                    x86_64  1.7.10-8.el9       appstream             39 k
 libXinerama              x86_64  1.1.4-10.el9       appstream             14 k
 libXmu                   x86_64  1.1.3-8.el9        appstream             74 k
 libXrandr                x86_64  1.5.2-8.el9        appstream             27 k
 libXrender               x86_64  0.9.10-16.el9      appstream             27 k
 libXt                    x86_64  1.2.0-6.el9        appstream            180 k
 libXxf86vm               x86_64  1.1.4-18.el9       appstream             18 k
 libdrm                   x86_64  2.4.121-1.el9      appstream            158 k
 libfontenc               x86_64  1.1.3-17.el9       appstream             30 k
 libglvnd                 x86_64  1:1.3.4-1.el9      appstream            133 k
 libglvnd-glx             x86_64  1:1.3.4-1.el9      appstream            140 k
 libjpeg-turbo            x86_64  2.0.90-7.el9       appstream            174 k
 libmpc                   x86_64  1.2.1-4.el9        appstream             61 k
 libpciaccess             x86_64  0.16-7.el9         baseos                26 k
 libpng                   x86_64  2:1.6.37-12.el9    baseos               116 k
 libtiff                  x86_64  4.4.0-13.el9       appstream            197 k
 libwayland-server        x86_64  1.21.0-1.el9       appstream             41 k
 libwebp                  x86_64  1.2.0-8.el9        appstream            276 k
 libxcb                   x86_64  1.13.1-9.el9       appstream            224 k
 libxkbfile               x86_64  1.1.0-8.el9        appstream             88 k
 libxshmfence             x86_64  1.3-10.el9         appstream             12 k
 llvm-libs                x86_64  18.1.8-3.el9       appstream             26 M
 mesa-dri-drivers         x86_64  24.1.2-3.el9       appstream            8.8 M
 mesa-filesystem          x86_64  24.1.2-3.el9       appstream             10 k
 mesa-libGL               x86_64  24.1.2-3.el9       appstream            169 k
 mesa-libgbm              x86_64  24.1.2-3.el9       appstream             35 k
 mesa-libglapi            x86_64  24.1.2-3.el9       appstream             44 k
 openh264                 x86_64  2.3.1-1.el9        epel-cisco-openh264  425 k
 pixman                   x86_64  0.40.0-6.el9_3     appstream            269 k
 tigervnc-license         noarch  1.14.1-1.el9_5     appstream             17 k
 tigervnc-server-minimal  x86_64  1.14.1-1.el9_5     appstream            1.2 M
 xkbcomp                  x86_64  1.4.4-4.el9        appstream             99 k
 xkeyboard-config         noarch  2.33-2.el9         appstream            779 k
 xorg-x11-server-utils    x86_64  7.7-44.el9         appstream            152 k
 xorg-x11-xauth           x86_64  1:1.1-10.el9       appstream             36 k
 xorg-x11-xinit           x86_64  1.4.0-11.el9       appstream             55 k
Installing weak dependencies:
 cpp                      x86_64  11.5.0-2.el9       appstream             11 M
 xrdp-selinux             x86_64  1:0.10.2-7.el9     epel                  13 k

Transaction Summary
================================================================================
Install  60 Packages

Total download size: 54 M
Installed size: 194 M
Is this ok [y/N]: n
Operation aborted.

The same problem doesn't arise on Rocky Linux.

@metalefty - I'm happy to try to raise this with Alma, but I suspect you'd rather do this. Let me know if you want me to take any further action.

@metalefty
Copy link
Member

Alma includes an epel-release package that doesn't contain /etc/yum.repos.d/epel-cisco-openh264.repo.

Ah, understood. That's definitely a Alma specific.

@thomaspmorgan
Copy link

Same issue with RHEL 9.5. I'll open a ticket with Redhat support. But why does xrdp work if color bit depth is 24, but not 36? What is xrdp doing that requires openh264 in this case?

@metalefty
Copy link
Member

metalefty commented Jan 28, 2025

But why does xrdp work if color bit depth is 24

Actually, it is NOT working yet. Just xrdp does not try to use H.264 when bpp <= 24. Literally, you won't fail to use something you don't use.

That is because xrdp does not use GFX when bpp <= 24.

[WARN ] client requested gfx protocol with insufficient color depth

What is xrdp doing that requires openh264 in this case?

When the following conditions are met, xrdp tries to encode screen by OpenH264.

  • xrdp is compiled with --enable-openh264 option
  • client supports GFX/H.264 and requests 32 bpp color
  • H.264 is preferred than other codec in xrdp's gfx.toml

When noopenh264 is installed instead of openh264, xrdp fails to use H.264.

@metalefty
Copy link
Member

xrdp available from EPEL is compiled with --enable-openh264 flag. So the solution is to make OpenH264 library available from xrdp (install it). Alternatively, not using H.264 is a workaround but not a solution.

As a workaround, there are some methods not to use H.264. The server-side approach and the client-side one.

  • Server-side: disable H.264 (edit gfx.toml).
  • Client-side: request other codec (depends on client)

@metalefty
Copy link
Member

@matt335672 Do you think it is possible to detect that noopenh264 is installed instead of openh264 in xrdp? If possible, I'm thinking of disabling H.264 on the xrdp side at runtime when xrdp is compiled with --enable-openh264 but openh264 is not installed.

@matt335672
Copy link
Member

I had a think about this a day or two ago, before the reasons for the failure were clear. It's easy enough to detect noopenh264. Given the way the code is structured, it would be hard in that situation to fall back to x264, but relatively easy to fallback to GFX. Tomorrow, I'll look at a patch for discussion.

@metalefty
Copy link
Member

I don't think fallback to x264 is required. Fallback to GFX-RFX is sufficient. EPEL doesn't have x264 package so xrdp is not built with x264 and fallback to x264 is not possible even if we implement such fallback on the RHEL-derivative systems.

The AlmaLinux team just released the latest epel-release package containing the epel-cisco-openh264 repo.

It will be distributed to mirrors within a day. Then, dnf install epel-release && dnf install xrdp will bring the proper openh264 package.

@matt335672
Copy link
Member

Thanks @metalefty

PR pushed for initial consideration.

@thomaspmorgan - I can't comment on how this could happen with RHEL as we don't have access to it. Any other feedback you can add if you get anything useful from support would be useful - we might be missing something.

@hauihau
Copy link
Author

hauihau commented Jan 29, 2025

I have never installed epel-release before, I have just a epel9.repo file which I copy over and modify it if we upgrade to a new major release. I have no idea why they have an extra repository just for openh264 instead of integrating it in the epel repository. I have seen that Fedora itself also has a seperate repository for openh264.

I can confirm, that after switching from noopenh264 to openh264, 32 bit color depth is working.

Just for my understanding: for color depth <32, H264 isn't used so there is no issue. Only for 32 bits H264 is used and fails, if noopenh264 is installed.

Issue can be closed as it is working now for me.

@hauihau
Copy link
Author

hauihau commented Jan 29, 2025

If you are in the situation where xrdp has been installed along with noopenh264, proceed as follows:-

  1. Check what depends on noopenh264 by trying to remove it:-

    $ sudo dnf --assumeno remove noopenh264
    Dependencies resolved.
    ================================================================================
     Package                  Arch    Version                     Repository   Size
    ================================================================================
    Removing:
     noopenh264               x86_64  0.1.0~openh264_2.4.1-2.el9  @epel        43 k
    Removing dependent packages:
     xorgxrdp                 x86_64  0.10.3-5.el9                @epel       178 k
     xrdp                     x86_64  1:0.10.2-7.el9              @epel       3.4 M
    Removing unused dependencies:
     dbus-x11                 x86_64  1:1.12.20-8.el9             @appstream   36 k
     fuse3-libs               x86_64  3.10.2-9.el9                @appstream  281 k
     imlib2                   x86_64  1.7.4-1.el9                 @epel       744 k
     tigervnc-license         noarch  1.14.1-1.el9_5              @appstream   18 k
     tigervnc-server-minimal  x86_64  1.14.1-1.el9_5              @appstream  2.8 M
     xrdp-selinux             x86_64  1:0.10.2-7.el9              @epel       9.0 k
    
    Transaction Summary
    ================================================================================
    Remove  9 Packages
    
    Freed space: 7.5 M
    Operation aborted.
    

    The above example shows only xrdp and xorgxrdp are affected. So it's safe to proceed.

  2. Manually kill any active xrdp sessions.

  3. Make sure epel-release is up to date with sudo dnf update epel-release ; sudo dnf update

  4. Replace noopenh264 with openh264 with sudo dnf install openh264 --allowerasing

  5. Restart xrdp with sudo systemctl restart xrdp
    You can use dnf swap noopenh264 openh264, swap is intended for such cases where you want to switch the provider of a library or similar.

@metalefty
Copy link
Member

I have never installed epel-release before, I have just a epel9.repo file which I copy over and modify it if we upgrade to a new major release.

I don't think it is a very good way to enable EPEL but it's your decision. The old repo file may cause a problem like what happened on AlmaLinux. I think this is a better way but it's up to you.

dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm 

I have no idea why they have an extra repository just for openh264 instead of integrating it in the epel repository.

This is due to the patent issue. Cisco covers patent royalties for the use of the binary form of the OpenH264 library. The OpenH264 library available in EPEL is provided by Cisco so end-users can use H.264 at no cost. I suspect there may be an agreement on redistribution between Fedora and Cisco that is what is causing the separate repositories.

See also: https://en.opensuse.org/OpenH264#Background

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants