Build failure: playwright-driver.browsers

[tobiasBora]

Nixpkgs version

• Unstable (25.11)

Steps to reproduce

playwright-driver.browser recently broke (and therefore break my PR #425503):

$ nix build .#playwright-driver.browsers

Can Hydra reproduce this build failure?

Hydra times out on https://hydra.nixos.org/search?query=playwright-browsers (NixOS/hydra#901)

Link to Hydra build job

No response

Relevant log output

$ nix build .#playwright-driver.browsers
warning: Git tree '/home/leo/nixpkgs' is dirty
error: builder for '/nix/store/ismn0nimsjw77yca3bds3ibnxz5s8nf6-libjxl-0.8.2.drv' failed with exit code 2;
       last 25 log lines:
       > /nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: libjxl.a(frame_header.cc.o):(.rodata+0x4): multiple definition of `jxl::YCbCrChromaSubsampling::kHShift'; CMakeFiles/encode_test.dir/jxl/encode_test.cc.o:(.rodata._ZN3jxl22YCbCrChromaSubsampling7kHShiftE[_ZN3jxl22YCbCrChromaSubsampling7kHShiftE]+0x0): first defined here
       > /nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: libjxl.a(frame_header.cc.o):(.rodata+0x0): multiple definition of `jxl::YCbCrChromaSubsampling::kVShift'; CMakeFiles/encode_test.dir/jxl/encode_test.cc.o:(.rodata._ZN3jxl22YCbCrChromaSubsampling7kVShiftE[_ZN3jxl22YCbCrChromaSubsampling7kVShiftE]+0x0): first defined here
       > collect2: error: ld returned 1 exit status
       > make[2]: *** [lib/CMakeFiles/encode_test.dir/build.make:124: lib/tests/encode_test] Error 1
       > make[1]: *** [CMakeFiles/Makefile2:3147: lib/CMakeFiles/encode_test.dir/all] Error 2
       > [ 82%] Linking CXX executable tests/fast_math_test
       > [ 82%] Built target fast_math_test
       > [ 83%] Linking CXX executable tests/decode_test
       > /nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: libjxl.a(frame_header.cc.o):(.rodata+0x4): multiple definition of `jxl::YCbCrChromaSubsampling::kHShift'; CMakeFiles/decode_test.dir/jxl/decode_test.cc.o:(.rodata._ZN3jxl22YCbCrChromaSubsampling7kHShiftE[_ZN3jxl22YCbCrChromaSubsampling7kHShiftE]+0x0): first defined here
       > /nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: libjxl.a(frame_header.cc.o):(.rodata+0x0): multiple definition of `jxl::YCbCrChromaSubsampling::kVShift'; CMakeFiles/decode_test.dir/jxl/decode_test.cc.o:(.rodata._ZN3jxl22YCbCrChromaSubsampling7kVShiftE[_ZN3jxl22YCbCrChromaSubsampling7kVShiftE]+0x0): first defined here
       > collect2: error: ld returned 1 exit status
       > make[2]: *** [lib/CMakeFiles/decode_test.dir/build.make:124: lib/tests/decode_test] Error 1
       > make[1]: *** [CMakeFiles/Makefile2:3033: lib/CMakeFiles/decode_test.dir/all] Error 2
       > [ 83%] Linking CXX executable tests/fast_dct_test
       > [ 83%] Built target fast_dct_test
       > [ 83%] Linking CXX executable tests/dct_test
       > /nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: libjxl.a(dct_scales.cc.o):(.rodata+0x380): multiple definition of `jxl::WcMultipliers<32ul>::kMultipliers'; CMakeFiles/dct_test.dir/jxl/dct_test.cc.o:(.rodata._ZN3jxl13WcMultipliersILm32EE12kMultipliersE[_ZN3jxl13WcMultipliersILm32EE12kMultipliersE]+0x0): first defined here
       > /nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: libjxl.a(dct_scales.cc.o):(.rodata+0x3c0): multiple definition of `jxl::WcMultipliers<16ul>::kMultipliers'; CMakeFiles/dct_test.dir/jxl/dct_test.cc.o:(.rodata._ZN3jxl13WcMultipliersILm16EE12kMultipliersE[_ZN3jxl13WcMultipliersILm16EE12kMultipliersE]+0x0): first defined here
       > /nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: libjxl.a(dct_scales.cc.o):(.rodata+0x3e0): multiple definition of `jxl::WcMultipliers<8ul>::kMultipliers'; CMakeFiles/dct_test.dir/jxl/dct_test.cc.o:(.rodata._ZN3jxl13WcMultipliersILm8EE12kMultipliersE[_ZN3jxl13WcMultipliersILm8EE12kMultipliersE]+0x0): first defined here
       > /nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: libjxl.a(frame_header.cc.o):(.rodata+0x4): multiple definition of `jxl::YCbCrChromaSubsampling::kHShift'; CMakeFiles/dct_test.dir/jxl/dct_test.cc.o:(.rodata._ZN3jxl22YCbCrChromaSubsampling7kHShiftE[_ZN3jxl22YCbCrChromaSubsampling7kHShiftE]+0x0): first defined here
       > /nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: libjxl.a(frame_header.cc.o):(.rodata+0x0): multiple definition of `jxl::YCbCrChromaSubsampling::kVShift'; CMakeFiles/dct_test.dir/jxl/dct_test.cc.o:(.rodata._ZN3jxl22YCbCrChromaSubsampling7kVShiftE[_ZN3jxl22YCbCrChromaSubsampling7kVShiftE]+0x0): first defined here
       > collect2: error: ld returned 1 exit status
       > make[2]: *** [lib/CMakeFiles/dct_test.dir/build.make:124: lib/tests/dct_test] Error 1
       > make[1]: *** [CMakeFiles/Makefile2:2995: lib/CMakeFiles/dct_test.dir/all] Error 2
       > make: *** [Makefile:146: all] Error 2
       For full logs, run:
         nix log /nix/store/ismn0nimsjw77yca3bds3ibnxz5s8nf6-libjxl-0.8.2.drv
error: 1 dependencies of derivation '/nix/store/2k7ks067gdady7jqq8khdv64pqjkmz43-playwright-webkit.drv' failed to build
error: 1 dependencies of derivation '/nix/store/z0g90qayl557jqwdk7x53jiy29pasawd-playwright-browsers.drv' failed to build

Additional context

No response

System metadata

Run 82f8090

Notify maintainers

@kalekseev @NyCodeGHG @nh2

---

Note for maintainers: Please tag this issue in your pull request description. (i.e. Resolves #ISSUE.)

I assert that this issue is relevant for Nixpkgs

• I assert that this is a bug and not a support request.
• I assert that this is not a duplicate of an existing issue.
• I assert that I have read the NixOS Code of Conduct and agree to abide by it.

Is this issue important to you?

Add a 👍 reaction to issues you find important.

[tobiasBora]

@kalekseev I think the issue comes from the libjxl' override introduced in a09e851 Do you know why we need this override and why we can't use the original libjxl definition?

[kalekseev]

@tobiasBora webkit needs old version of libjxl. To debug the problem you should probably start from running git bisect between broken and working commits and find exact commit that introduced breaking change. It might be some change in libjxl derviation that override in pw driver does not handle properly.

[lheckemann]

FWIW I didn't find a hydra job for playwright-driver.browsers itself, but https://hydra.nixos.org/job/nixpkgs/trunk/aider-chat-with-playwright.x86_64-linux is a job that fails because of this.

Failure was introduced in the staging merge b47d4f0, this will take a while to bisect...

[tobiasBora]

@kalekseev is it just because you rely on the pre-build version that is build with libjxl, or due to a fundamental issue is recent versions of libjxl?

So I tried to copy the definition of libjxl that was existing in the latest upgrade a09e851, giving:

{
  lib,
  stdenv,
  fetchzip,
  fetchFromGitHub,
  makeWrapper,
  autoPatchelfHook,
  patchelfUnstable,
  fetchpatch,
  libjxl,
  brotli,
  asciidoc,
  at-spi2-atk,
  cmake,
  cairo,
  doxygen,
  flite,
  fontconfig,
  freetype,
  gdk-pixbuf,
  giflib,
  glib,
  glib-networking,
  gperftools,
  graphviz,
  gst_all_1,
  gtest,
  harfbuzz,
  harfbuzzFull,
  icu70,
  lcms,
  lcms2,
  libavif,
  libdrm,
  libepoxy,
  libevent,
  libgcc,
  libgcrypt,
  libgpg-error,
  libhwy,
  libjpeg,
  libjpeg8,
  libopus,
  libpng,
  libsoup_3,
  libtasn1,
  libvpx,
  libwebp,
  libwpe,
  libwpe-fdo,
  libxkbcommon,
  libxml2,
  libxslt,
  libgbm,
  openexr_3,
  pkg-config,
  python3,
  sqlite,
  systemdLibs,
  wayland-scanner,
  woff2,
  zlib,
  suffix,
  revision,
  system,
  throwSystem,
}:
let
  suffix' =
    if lib.hasPrefix "linux" suffix then
      "ubuntu-22.04" + (lib.removePrefix "linux" suffix)
    else if lib.hasPrefix "mac" suffix then
      "mac-14" + (lib.removePrefix "mac" suffix)
    else
      suffix;
  libvpx' = libvpx.overrideAttrs (
    finalAttrs: previousAttrs: {
      version = "1.12.0";
      src = fetchFromGitHub {
        owner = "webmproject";
        repo = finalAttrs.pname;
        rev = "v${finalAttrs.version}";
        sha256 = "sha256-9SFFE2GfYYMgxp1dpmL3STTU2ea1R5vFKA1L0pZwIvQ=";
      };
    }
  );
  libavif' = libavif.overrideAttrs (
    finalAttrs: previousAttrs: {
      version = "0.9.3";
      src = fetchFromGitHub {
        owner = "AOMediaCodec";
        repo = finalAttrs.pname;
        rev = "v${finalAttrs.version}";
        hash = "sha256-ME/mkaHhFeHajTbc7zhg9vtf/8XgkgSRu9I/mlQXnds=";
      };
      postPatch = "";
      patches = [ ];
    }
  );

  libjxl' = stdenv.mkDerivation rec {
    pname = "libjxl";
    version = "0.8.2";

    outputs = [
      "out"
      "dev"
    ];

    src = fetchFromGitHub {
      owner = "libjxl";
      repo = "libjxl";
      rev = "v${version}";
      hash = "sha256-I3PGgh0XqRkCFz7lUZ3Q4eU0+0GwaQcVb6t4Pru1kKo=";
      fetchSubmodules = true;
    };

    strictDeps = true;

    patches = [
      # Add missing <atomic> content to fix gcc compilation for RISCV architecture
      # https://github.com/libjxl/libjxl/pull/2211
      (fetchpatch {
        url = "https://github.com/libjxl/libjxl/commit/22d12d74e7bc56b09cfb1973aa89ec8d714fa3fc.patch";
        hash = "sha256-X4fbYTMS+kHfZRbeGzSdBW5jQKw8UN44FEyFRUtw0qo=";
      })
    ];


    nativeBuildInputs = [
      cmake
      pkg-config
      gdk-pixbuf
      makeWrapper
      asciidoc
      doxygen
      python3
    ];

    depsBuildBuild = [
      graphviz
    ];

    # Functionality not currently provided by this package
    # that the cmake build can apparently use:
    #     OpenGL/GLUT (for Examples -> comparison with sjpeg)
    #     viewer (see `cmakeFlags`)
    #     plugins like for GDK and GIMP (see `cmakeFlags`)

    # Vendored libraries:
    # `libjxl` currently vendors many libraries as git submodules that they
    # might patch often (e.g. test/gmock, see
    # https://github.com/NixOS/nixpkgs/pull/103160#discussion_r519487734).
    # When it has stabilised in the future, we may want to tell the build
    # to use use nixpkgs system libraries.

    # As of writing, libjxl does not point out all its dependencies
    # conclusively in its README or otherwise; they can best be determined
    # by checking the CMake output for "Could NOT find".
    buildInputs = [
      lcms2
      giflib
      gperftools # provides `libtcmalloc`
      gtest
      libjpeg
      libpng
      libwebp
      gdk-pixbuf
      openexr_3
      zlib
    ];

    propagatedBuildInputs = [
      brotli
      libhwy
    ];

    cmakeFlags =
      [
        # For C dependencies like brotli, which are dynamically linked,
        # we want to use the system libraries, so that we don't have to care about
        # installing their .so files generated by this build.
        # The other C++ dependencies are statically linked in, so there
        # using the vendorered ones is easier.
        "-DJPEGXL_FORCE_SYSTEM_BROTLI=ON"

        # Use our version of highway, though it is still statically linked in
        "-DJPEGXL_FORCE_SYSTEM_HWY=ON"

        # Use our version of gtest
        "-DJPEGXL_FORCE_SYSTEM_GTEST=ON"
      ]
      ++ lib.optionals stdenv.hostPlatform.isStatic [
        "-DJPEGXL_STATIC=ON"
      ]
      ++ lib.optionals stdenv.hostPlatform.isAarch32 [
        "-DJPEGXL_FORCE_NEON=ON"
      ];

    CXXFLAGS = lib.optionalString stdenv.hostPlatform.isAarch32 "-mfp16-format=ieee";

    # FIXME x86_64-darwin:
    # https://github.com/NixOS/nixpkgs/pull/204030#issuecomment-1352768690
    doCheck = with stdenv; !(hostPlatform.isi686 || isDarwin && isx86_64);

    meta = with lib; {
      homepage = "https://github.com/libjxl/libjxl";
      description = "JPEG XL image format reference implementation";
      license = licenses.bsd3;
      maintainers = with maintainers; [ nh2 ];
      platforms = platforms.all;
    };
  };

  webkit-linux = stdenv.mkDerivation {
    name = "playwright-webkit";
    src = fetchzip {
      url = "https://playwright.azureedge.net/builds/webkit/${revision}/webkit-${suffix'}.zip";
      stripRoot = false;
      hash =
        {
          x86_64-linux = "sha256-lwH783B3/laqw0IdGBnVzvySRoF0AwZsSolaqUKmsM4=";
          aarch64-linux = "sha256-qtvP0bc5rcZcz6SqigfdrjhTWEmvT4k11I1GW1Eoj/Q=";
        }
        .${system} or throwSystem;
    };

    nativeBuildInputs = [
      autoPatchelfHook
      patchelfUnstable
      makeWrapper
    ];
    buildInputs = [
      at-spi2-atk
      cairo
      flite
      fontconfig.lib
      freetype
      glib
      brotli
      libjxl'
      gst_all_1.gst-plugins-bad
      gst_all_1.gst-plugins-base
      gst_all_1.gstreamer
      harfbuzz
      harfbuzzFull
      icu70
      lcms
      libavif'
      libdrm
      libepoxy
      libevent
      libgcc.lib
      libgcrypt
      libgpg-error
      libjpeg8
      libopus
      libpng
      libsoup_3
      libtasn1
      libwebp
      libwpe
      libwpe-fdo
      libvpx'
      libxml2
      libxslt
      libgbm
      sqlite
      systemdLibs
      wayland-scanner
      woff2.lib
      libxkbcommon
      zlib
    ];

    patchelfFlags = [ "--no-clobber-old-sections" ];
    buildPhase = ''
      cp -R . $out

      # remove unused gtk browser
      rm -rf $out/minibrowser-gtk
      # remove bundled libs
      rm -rf $out/minibrowser-wpe/sys

      # TODO: still fails on ubuntu trying to find libEGL_mesa.so.0
      wrapProgram $out/minibrowser-wpe/bin/MiniBrowser \
        --prefix GIO_EXTRA_MODULES ":" "${glib-networking}/lib/gio/modules/" \
        --prefix LD_LIBRARY_PATH ":" $out/minibrowser-wpe/lib

    '';

    preFixup = ''
      # Fix libxml2 breakage. See https://github.com/NixOS/nixpkgs/pull/396195#issuecomment-2881757108
      mkdir -p "$out/lib"
      ln -s "${lib.getLib libxml2}/lib/libxml2.so" "$out/lib/libxml2.so.2"
    '';
  };
  webkit-darwin = fetchzip {
    url = "https://playwright.azureedge.net/builds/webkit/${revision}/webkit-${suffix'}.zip";
    stripRoot = false;
    hash =
      {
        x86_64-darwin = "sha256-p1+Pk+Zhf2OPEmEWCEd0tA7CdoMcOgYp69SnQXufFJ0=";
        aarch64-darwin = "sha256-tEfKvJuGe4htZLSOn94eKeBtWXYkjl73iJSY4BWJMKo=";
      }
      .${system} or throwSystem;
  };
in
{
  x86_64-linux = webkit-linux;
  aarch64-linux = webkit-linux;
  x86_64-darwin = webkit-darwin;
  aarch64-darwin = webkit-darwin;
}
.${system} or throwSystem

but I still get the exact same error… So maybe it has to do with a change in more fundamental level, e.g. gcc/ld/…