-
Notifications
You must be signed in to change notification settings - Fork 207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
avoid hardcoding /scratch/
#25
base: main
Are you sure you want to change the base?
Conversation
Hi ! Thank you for your PR :) we've had some really annoying issues when leaving tmp to be the system default. Especially on clusters used by many people where /tmp fills up fast. Although the code doesn't directly use TMP_DIR, there are many packages beneath that do, most importantly multiprocessing and triton. A full tmp makes spawning processes extremely slow and painful and makes the whole code slower silently. How about making that environment variable configurable ? |
Ah, I see. Forgive me then, I didn't realize the code was harmless when the folder was missing. |
if the intended target is |
My bad ! Triton doesn't directly use it, we have this in our env setup also Lines 212 to 220 in 60af836
But we do that before setting temp_dir so triton is probably not affected by that ! Let me see how to make this better |
on slurm jobs, a
/scratch/slurm_tmpdir/
folder is attempted to be used for theTMP_DIR
env var.this is strange, given that
TMP_DIR
is not used anywhere else in the repository, nor is it a common env var name/scratch/
does not exist in all slurm setups, and is an artefact of the author's personal cluster having their NFS at/scratch/
specificallytherefore I propose deleting the code