[FEA] Expose CUDA 13 async pools for managed and pinned memory

[bdice]

Is your feature request related to a problem? Please describe.
CUDA 13 added APIs like this:

​cudaError_t cudaMemGetDefaultMemPool(cudaMemPool_t* memPool, cudaMemLocation* location, cudaMemAllocationType type)

(docs) This API can be used to get a cudaMemPool_t that is backed by managed or pinned memory. This can be coupled with cudaMallocFromPoolAsync to have an async (stream-ordered) allocator for managed or pinned memory.

This would allow applications to stop using RMM's pool adaptor around a managed MR, which has lots of awkward behavior. In general, the driver pool should have better performance and better defragmentation than RMM's own pool adaptor implementation.

Describe the solution you'd like
There are several ways this API could look. I am showing the managed MR for now but a similar thing can be done for pinned.

• (default choice) We could add a new async_managed_memory_resource
  • Provides a clear way to say "this is a new feature! use this!"
• We could refactor managed_memory_resource to use the async allocator in CUDA 13
  • Provides some immediate benefit to existing users with no code change
  • This could be done in addition to a new async_managed_memory_resource class? No obvious downsides that I can see.

Describe alternatives you've considered

• We could add a parameter to the async_memory_resource constructor, like type="managed"
  • This would probably be overlooked in favor of the managed_memory_resource, even if we document it extensively
• We could encourage users to get their own cudaMemPool_t handle and use async_view_memory_resource
  • Awkward because it requires direct CUDA runtime interactions, and it's more code than most RMM users are used to.

Additional context
async_managed_memory_resource should raise an error on construction with CUDA 12, since it's not supported there.

For async_memory_resource today, we construct a special cudaMemPool_t with flags to enable the decompression engine. I think decompression engine isn't supported with managed memory, so the default managed memory pool should be enough here? This should be checked.

[vyasr]

At least for pinned memory it looks like we might be able to solve this already in CUDA 12, we just have to create our own pool instead of using the default one provided by the driver. From the docs it seems like cudaMemAllocationTypePinned and cudaMallocFromPoolAsync were already available in CUDA 12.0. If the main new addition in this space in CUDA 13 is the existence of default driver managed pools (i.e. cudaMemGetDefaultMemPool), then we could probably support both CUDA 12 and CUDA 13 for pinned memory today by creating a pool manually. That would fix #2053.

[vyasr]

OK, after some further research it looks like this cannot be done until CUDA 12.6. Pinned memory pools and asynchronous allocation was not supported until then. The pinned memory type prior to that could only be used for on-device allocations. So I don't think we have an option for #2053 until CUDA 13.

[bdice]

@vyasr I feel like RMM should probably try to enable something like this for CUDA >=12.6,<13.0. Do you agree? Can you be specific about what APIs you would use here for that case? (CUDA docs links preferred)

[vyasr]

Whoops my previous message was supposed to include links but I didn't copy them in. Yes, I'm basically just proposing that we do a cudaMemPoolCreate with a memory type of pinned and a location of host when the pinned mr is created and then do all allocations with cudaMallocFromPoolAsync from that pool. On CUDA 13 you can replace cudaMemPoolCreate with cudaMemGetDefaultMemPool so that you also share the pool with all other pinned memory allocations in the process.

Aside: For CUDA 13 it might be more correct to use cudaMemGetMemPool than cudaMemGetDefaultMemPool so that if somebody has set the mem pool for a device we use it. Alternatively, the rmm mr could accept an option on construction to indicate which of those pools to prefer.