[C++] Nightly verification jobs fail on Ubuntu 24.04 to build byte_stream_split_internal.cc due to XSIMD failure

[raulcd]

Describe the bug, including details regarding any error messages, version, and platform.

Some of our nightly jobs are having compilation errors with xsimd on byte_stream_split_internal.cc:

[360/1273] Building CXX object src/arrow/CMakeFiles/arrow_util.dir/util/byte_stream_split_internal.cc.o
FAILED: src/arrow/CMakeFiles/arrow_util.dir/util/byte_stream_split_internal.cc.o 
/usr/bin/c++ -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_AVX512 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_WITH_TIMING_TESTS -DBOOST_ALL_NO_LIB -DBOOST_CONTAINER_NO_LIB -DBOOST_CONTAINER_STATIC_LINK -DBOOST_CONTEXT_EXPORT="" -DBOOST_CONTEXT_NO_LIB="" -DBOOST_CONTEXT_STATIC_LINK="" -DBOOST_DATE_TIME_NO_LIB -DBOOST_DATE_TIME_STATIC_LINK -DBOOST_MPL_CFG_NO_PREPROCESSED_HEADERS -DBOOST_RANDOM_NO_LIB -DBOOST_RANDOM_STATIC_LINK -DURI_STATIC_BUILD -DXSIMD_ENABLE_XTL_COMPLEX=1 -I/tmp/arrow-HEAD.gEvsY/cpp-build/src -I/arrow/cpp/src -I/arrow/cpp/src/generated -I/tmp/arrow-HEAD.gEvsY/cpp-build/_deps/lz4-src/build/cmake/../../lib -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/headers/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/algorithm/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/array/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/assert/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/config/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/static_assert/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/throw_exception/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/bind/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/core/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/concept_check/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/preprocessor/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/type_traits/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/exception/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/smart_ptr/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/tuple/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/function/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/iterator/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/detail/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/fusion/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/container_hash/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/describe/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/mp11/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/function_types/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/mpl/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/predef/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/utility/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/io/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/typeof/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/functional/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/optional/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/range/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/conversion/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/regex/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/unordered/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/align/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/asio/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/context/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/pool/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/integer/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/winapi/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/date_time/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/lexical_cast/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/container/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/intrusive/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/move/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/numeric/conversion/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/tokenizer/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/system/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/variant2/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/crc/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/dynamic_bitset/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/math/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/random/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/multiprecision/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/ratio/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/scope/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/scope_exit/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/type_index/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/_deps/boost-src/libs/uuid/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/brotli_ep/src/brotli_ep-install/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/bzip2_ep-install/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/snappy_ep/src/snappy_ep-install/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/rapidjson_ep/src/rapidjson_ep-install/include -isystem /tmp/arrow-HEAD.gEvsY/cpp-build/zstd_ep-install/include -Wno-noexcept-type -Wno-self-move  -fdiagnostics-color=always  -Wall -fno-semantic-interposition -msse4.2  -O3 -DNDEBUG -O2 -ftree-vectorize  -std=c++17 -fPIC -MD -MT src/arrow/CMakeFiles/arrow_util.dir/util/byte_stream_split_internal.cc.o -MF src/arrow/CMakeFiles/arrow_util.dir/util/byte_stream_split_internal.cc.o.d -o src/arrow/CMakeFiles/arrow_util.dir/util/byte_stream_split_internal.cc.o -c /arrow/cpp/src/arrow/util/byte_stream_split_internal.cc
In file included from /arrow/cpp/src/arrow/util/byte_stream_split_internal.cc:18:
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h: In function 'xsimd::batch<signed char, A> arrow::util::internal::zip_lo_n(const xsimd::batch<signed char, A>&, const xsimd::batch<signed char, A>&)':
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: error: type/value mismatch at argument 2 in template parameter list for 'template<class batch_type, typename batch_type::value_type ...Values> struct xsimd::batch_constant'
  141 |                        xsimd::batch_constant<uint64_t, Arch, 0, 1, 4, 5>{}));
      |                                                                        ^
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: note:   expected a constant of type 'typename batch_type::value_type', got 'Arch'
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: error: 'long unsigned int' is not a class, struct, or union type
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: note: invalid template non-type parameter
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: error: 'long unsigned int' is not a class, struct, or union type
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: note: invalid template non-type parameter
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: error: 'long unsigned int' is not a class, struct, or union type
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: note: invalid template non-type parameter
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: error: 'long unsigned int' is not a class, struct, or union type
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:141:72: note: invalid template non-type parameter
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h: In function 'xsimd::batch<signed char, A> arrow::util::internal::zip_hi_n(const xsimd::batch<signed char, A>&, const xsimd::batch<signed char, A>&)':
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: error: type/value mismatch at argument 2 in template parameter list for 'template<class batch_type, typename batch_type::value_type ...Values> struct xsimd::batch_constant'
  165 |                        xsimd::batch_constant<uint64_t, Arch, 2, 3, 6, 7>{}));
      |                                                                        ^
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: note:   expected a constant of type 'typename batch_type::value_type', got 'Arch'
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: error: 'long unsigned int' is not a class, struct, or union type
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: note: invalid template non-type parameter
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: error: 'long unsigned int' is not a class, struct, or union type
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: note: invalid template non-type parameter
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: error: 'long unsigned int' is not a class, struct, or union type
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: note: invalid template non-type parameter
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: error: 'long unsigned int' is not a class, struct, or union type
/arrow/cpp/src/arrow/util/byte_stream_split_internal.h:165:72: note: invalid template non-type parameter

Nightly failures:

• verify-rc-source-cpp-linux-ubuntu-24.04-amd64
• verify-rc-source-integration-linux-ubuntu-24.04-amd64
• verify-rc-source-python-linux-ubuntu-24.04-amd64
• verify-rc-source-ruby-linux-ubuntu-24.04-amd64

Potentially related with the version of xsimd used on those jobs.

Component(s)

C++

[raulcd]

cc @AntoinePrv

[raulcd]

It seems we might have to use xsimd_SOURCE=BUNDLED as Ubuntu 24.04 uses libxsimd-dev 12.1.1 but we require minimum 13.0.0

[kou]

Ah, we should have changed required xsimd version in #46963.

diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index 722b3b56de..04c136b304 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake
+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake
@@ -2549,7 +2549,7 @@ if(ARROW_USE_XSIMD)
                      IS_RUNTIME_DEPENDENCY
                      FALSE
                      REQUIRED_VERSION
-                     "8.1.0")
+                     "13.0.0")
 
   if(xsimd_SOURCE STREQUAL "BUNDLED")
     set(ARROW_XSIMD arrow::xsimd)

[AntoinePrv]

Yes, I think we should use 13.0, as per @pitrou comment, 12.1 was failing in CI.
#46963 (comment)