DPDK with SR-IOV does not receive UL from RU

[yc541]

Hello,

We are testing a Benetel RAN650 (v1.4.0) connected to the DU via a Falcon PTP switch. The DU runs on a Dell server with Intel(R) Xeon(R) Gold 5420+ CPU, 56 cores. The NIC is 82599ES 10G, 2-port.

The PTP sync runs OK on both DU and RU, and we got the 100MHz 4x4 MIMO working (no DPDK) with UE attach, but with high PUSCH loss rate (around 20% to 50%). After checking the gNB logs, we can see there are UL ethernet packet losses (ecpri packet sequence id skipping) from the RU then we try to enable DPDK to improve the performance.

We have been running two RUs on the same 2-port NIC so we went for the SR-IOV route with VF. The VF is created using
echo 1 | sudo tee /sys/class/net/enp55s0f0/device/sriov_numvfs
sudo ip link set enp55s0f0 vf 0 mac 00:33:22:33:00:11 spoofchk off
sudo dpdk-devbind.py --bind=vfio-pci 0000:37:10.0
./dpdk-devbind.py -s shows

The VF's MAC address (00:33:22:33:00:11) is also configured in gnb config and ru_config.cfg on the Benetel.

Hal in gnb is set to:
hal: eal_args: "--lcores (0-1)@(0-47)"

When running 1 RU on the VF, all the DL packets arrive at the RU with no latency, and the RU is sending back UL (prach) to the DU

The UE sees the cell and sends PRACH but in gnb logs there are no prach received. So we checked the code in dpdk_receiver_impl::receive() and num_frames is always 0 which means the DU didn't get any UL packets from RU (although the RU clearly sends UL packets).

I saw a few other threads #1118 #1317 mentioning DL not reaching RU but somehow we got the opposite. Any idea how can we debug this issue? Many thanks!