Skip to content

Allow specifying custom tempdir for files.download #1366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: 3.x
Choose a base branch
from

Conversation

scy
Copy link
Contributor

@scy scy commented Jun 2, 2025

This introduces a new argument to the files.download operation: temp_dir, which allows the user to specify a custom temporary directory for downloading into.

I know that TEMP_DIR exists as an (undocumented) config option, but since Host._get_temp_directory is memoized, it's not feasible to set this option only for certain operations.

My use case is downloading very large files (> 100 GB) into specific volumes on the server, which are backed by different filesystems and have some constraints regarding free space. The default of downloading everything into /tmp (or whatever) doesn't work because it would fill up the root partition before the download even finishes, but setting TEMP_DIR to different directories throughout a single deploy is also not really possible without resorting to hacks, afaics.

To allow customizing the temporary directory, Host.get_temp_filename also gained a (keyword-only, optional) new argument temp_directory.

@scy
Copy link
Contributor Author

scy commented Jun 2, 2025

The failing tests are unrelated to my changes.

@Fizzadar Fizzadar force-pushed the download-tempdir branch from c43638d to ed8550d Compare June 6, 2025 08:33
@Fizzadar Fizzadar force-pushed the download-tempdir branch from ed8550d to 6803928 Compare June 8, 2025 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant