prov/efa: build failure on s390/ppc64 #11389
Description
Describe the bug
Compilation of libfabric 2.3.0rc1 fails on both s390 and ppc64
On s390:
[ 163s] libtool: compile: gcc -DHAVE_CONFIG_H -I. -I./include -D_GNU_SOURCE -D__USE_XOPEN2K8 -DSYSCONFDIR=\"/etc\" -DRDMADIR=\"@rdmadir@\" -DPROVDLDIR=\"/usr/lib64/libfabric\" -I./prov/sockets/include -I./prov/sockets -I./prov/lpp/include -I./prov/lpp -I./prov/verbs/include -I./prov/verbs/include/linux -I./prov/verbs/src/ep_rdm -I./prov/verbs/src/ep_dgram -I./prov/efa/src/ -I./prov/efa/src/rdm/ -D__LIBUSNIC__ -DWANT_DEBUG_MSGS=0 -DHAVE_LIBNL3=1 -I/usr/include/libnl3 -I./prov/usnic/src/usnic_direct -I./prov/lnx/include -I./prov/hook/include -I./prov/hook/perf/include -I./prov/hook/monitor/include -I./prov/hook/hook_debug/include -I./prov/hook/hook_hmem/include -I./prov/hook/dmabuf_peer_mem/include -Wall -O2 -DNDEBUG -Wno-incompatible-pointer-types --std=gnu17 -c prov/efa/src/efa_shm.c -fPIC -DPIC -o prov/efa/src/.libs/src_libfabric_la-efa_shm.o
[ 163s] In file included from prov/efa/src/efa_data_path_direct_structs.h:34,
[ 163s] from prov/efa/src/efa_base_ep.h:15,
[ 163s] from prov/efa/src/efa.h:39,
[ 163s] from prov/efa/src/efa_shm.c:5:
[ 163s] prov/efa/src/efa_io_defs.h:426:10: fatal error: s390_mmio_insn.h: No such file or directory
On PPC64:
[ 153s] libtool: link: gcc -Wall -O2 -DNDEBUG -Wno-incompatible-pointer-types --std=gnu17 -flto=auto -o util/.libs/fi_strerror util/strerror.o src/.libs/libfabric.so -lrdmacm -lefa -libverbs -lnl-3 -lnl-route-3 -lucp -luct -lucm -lucs -latomic -lpthread -ldl -Wl,-rpath -Wl,/usr/lib64
[ 153s] /usr/lib64/gcc/powerpc64le-suse-linux/15/../../../../powerpc64le-suse-linux/bin/ld: src/.libs/libfabric.so: undefined reference to `mmio_write64_be'
This is due to 741ca49 ("prov/efa: Bypass rdma-core in data path.")
It copied a file from rdma-core internal headers but lacks some surrounding code for these 2 archs.
@shijin-aws Is EFA supported on non x86_64 arch ?
If not, scrubbing the arch specific code in the header + disabling the provider on these would be a simple fix.
If yes, there's quite a few more functions needed to bec picked from rdma-core to get this to work.