Skip to content

Mempool memory resource - IPC #446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 29 commits into
base: main
Choose a base branch
from
Draft

Conversation

ksimpson-work
Copy link
Contributor

@ksimpson-work ksimpson-work commented Feb 12, 2025

This is ready for at least a first look

Add Mempool class as the first public MemoryResource implementation. It supports IPC.

Add IPC buffer which is an internal buffer implementation used to represent a buffer descriptor which is used to export and import buffers across process boundaries. It implements reduce which is expected by common multiprocessing libraries. It's use is seen in the tests added to this review.

Copy link
Contributor

copy-pr-bot bot commented Feb 12, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ksimpson-work
Copy link
Contributor Author

/ok to test

@leofang leofang added triage Needs the team's attention P0 High priority - Must do! feature New feature or request cuda.core Everything related to the cuda.core module labels Feb 14, 2025
@ksimpson-work
Copy link
Contributor Author

/ok to test

Copy link

@ksimpson-work
Copy link
Contributor Author

/ok to test

@ksimpson-work ksimpson-work changed the title WIP: IPC Mempool memory resource - IPC Feb 24, 2025
@ksimpson-work
Copy link
Contributor Author

/ok to test

1 similar comment
@ksimpson-work
Copy link
Contributor Author

/ok to test

@ksimpson-work
Copy link
Contributor Author

/ok to test

@leofang leofang self-requested a review February 25, 2025 05:50
@ksimpson-work
Copy link
Contributor Author

/ok to test

@ksimpson-work
Copy link
Contributor Author

/ok to test

@ksimpson-work
Copy link
Contributor Author

/ok to test

@ksimpson-work
Copy link
Contributor Author

/ok to test

Comment on lines 22 to 35
# Check if pywin32 is available on Windows
_PYWIN32_AVAILABLE = False
if platform.system() == "Windows":
try:
import win32security

_PYWIN32_AVAILABLE = True
except ImportError:
import warnings

warnings.warn(
"pywin32 module not found. For better IPC support on Windows, " "install it with: pip install pywin32",
stacklevel=2,
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How hard would it be to recreate the functionality we need from here to avoid this optional dependency?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll need to discuss our next move first and then clean this up based on the decision. Keenan struggled for quite a long time and it turns out the IPC support for mempools is not enabled on Windows at all. There is a number of doc and sample bugs we'll create later (the memory team created one).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created 2 bugs to address the documentation issues.

yeah the review is in limbo right now. I'm not sure if we should push the change for IPC mempools and have it only support linux for now, or if we should shelve it and add a cuMem and VMM API IPC memory_resource in the meantime.

All of the Windows specific changes should be treated as draft. That optional dependency is from when I was debugging windows errors and operating under the assumption that I was improperly creating the windows security descriptor for the mempool handle. In hindsight it was a (temporary) lack of suport in driver layer as Leo said.

@ksimpson-work ksimpson-work marked this pull request as ready for review March 5, 2025 18:13
@ksimpson-work
Copy link
Contributor Author

/ok to test

@ksimpson-work
Copy link
Contributor Author

ksimpson-work commented Mar 6, 2025

I am tasked with adding the IPC mempool support for windows at the driver level (5151668), so I think we should roll with this implementation and have it pick up windows once that change is integrated. Users who need IPC on windows for older CTKs can implement their own MR or use the cuMemCreate & VMM API bindings. WDYT

@leofang leofang marked this pull request as draft May 2, 2025 04:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuda.core Everything related to the cuda.core module feature New feature or request P0 High priority - Must do! triage Needs the team's attention
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants