ASAN: use-after-free in mpi4py's TestP2PMatchedWorld.testIMProbe

[devreal]

Running mpi4py with current main under GCC's address sanitizer fails with this error:

testIMProbe (test_p2p_obj_matched.TestP2PMatchedWorld.testIMProbe) ... =================================================================
==4179551==ERROR: AddressSanitizer: heap-use-after-free on address 0x6150000702b8 at pc 0x7fffea3700c1 bp 0x7fffffff5b40 sp 0x7fffffff5b38
READ of size 8 at 0x6150000702b8 thread T0
    #0 0x7fffea3700c0 in ompi_request_check_same_instance ../../ompi/request/request.c:274
    #1 0x7fffea53cfcb in PMPI_Waitall /gpfs/projects/SchuchartGroup/src/openmpi/ompi/build/ompi/mpi/c/waitall_generated.c:66
    #2 0x7fffeb0cd4e0 in __pyx_f_6mpi4py_3MPI_PyMPI_waitall src/mpi4py/MPI.c:101369
    #3 0x7fffeb0cd4e0 in __pyx_pf_6mpi4py_3MPI_7Request_64waitall src/mpi4py/MPI.c:139235
    #4 0x7fffeb0cd4e0 in __pyx_pw_6mpi4py_3MPI_7Request_65waitall src/mpi4py/MPI.c:139200

The line is question is https://github.com/open-mpi/ompi/blob/main/ompi/request/request.c#L274 where we access the communicator to get the instance. Maybe we're missing a RETAIN/RELEASE on the communicator?

[hppritcha]

was this using OB1 PML?

[hppritcha]

looks like UCX pml isn't handling MPI_Imrecv correctly.