-
Notifications
You must be signed in to change notification settings - Fork 570
Use consistent names for internal nvcc
files
#2383
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Use consistent names for internal nvcc
files
#2383
Conversation
Looks like integration tests are failing due to https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/#decommissioned-cache-service-brownouts |
cc: @robertmaynard for review |
…ts can run in parallel
These changes ensure cache hits for compilations which are subsets of previously cached compilations * Normalize cudafe++, ptx, and cubin names regardless of whether the compilation flag is `-c`, `-ptx`, `-cubin`, or whether there are one or many `-gencode` flags * Include the compiler `hash_key` in the output dir for internal nvcc files to guarantee stability and uniqueness * Fix cache error due to hash collision from not hashing all the PTX and cubin flags
add more multi-arch tests to ensure combining cached/new PTX and cubins doesn't produce corrupted objects
…nding deprecation
fc85929
to
051fec9
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2383 +/- ##
==========================================
- Coverage 71.41% 69.57% -1.84%
==========================================
Files 65 45 -20
Lines 36349 26521 -9828
==========================================
- Hits 25960 18453 -7507
+ Misses 10389 8068 -2321 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
7ba3fae
to
ffe68b9
Compare
ffe68b9
to
3c08b18
Compare
Output(PathBuf), | ||
PassThrough(OsString), | ||
UnhashedFlag, | ||
ExtraOutput(PathBuf), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I renamed this from UnhashedOutput -> ExtraOutput
to reflect that these argument types are now hashed.
This PR extracts the fixes from #2356 per @drahnr's request.
This PR branched from/is a follow-up to #2382. See the diff of the two branches here.
The names for the internal files depend on the compilation flag and device architectures.
nvcc
generates a different name for the.cpp1.ii
,.cudafe1.c
,.cudafe1.stub.c
,.cudafe1.gpu
,.ptx
, and.cubin
files when the compile flag is-ptx
,-cubin
, or-c
, and also on whether there's one vs. many-gencode
arguments. Additionally, it will either include or omit the--gen_module_id_file
flag from thecicc
invocation based on whether the compile flag is-ptx
,-cubin
, or-c
.Some examples:
From the above, we observe that:
.cpp1.ii
,.cudafe1.c
,.cudafe1.stub.c
,.cudafe1.gpu
, and.ptx
files are either:x.<suffix>
x.compute_XX.<suffix>
.cubin
files are either:x.cubin
x.compute_XX.cubin
x.compute_XX.sm_XX.cubin
-c
, thecicc
command includes--gen_module_id_file
-c
, thecicc
command omits--gen_module_id_file
This PR hashes all the
cudafe++
,cicc
, andptxas
arguments to avoid collisions, but nvcc's inconsistent file naming leads to cache misses when there should be hits. So for simplicity I updated the renaming logic to rename to the longest form of each (i.e.x.compute_XX.ptx
,x.compute_XX.sm_XX.cubin
), and always add the--gen_module_id_file
flag tocicc
invocations.