Skip to content

Tracking Issue: Concurrent Downloads #18278

@reitermarkus

Description

@reitermarkus

Concurrent Downloads

  • Implement MVP of concurrent downloads for brew fetch.

    Implemented in Implement concurrent downloads in brew fetch. #17756.

  • Improve output logic.

    For simplicity, the output currently only uses at most terminal height - 1 lines due to a trailing newline.

    Implemented in Use full terminal height for concurrent output. #19194.

  • Handle custom download strategies. These may not be possible to run concurrently so let's handle them differently.

    Skipping these for now.

  • Implement graceful cancellation of downloads. Currently, cancelling downloads can only be done by killing the whole thread pool, i.e. the sledgehammer approach. Proper cancellation based on https://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/Cancellation.html should be implemented, making it possible to neatly show successful, failed and cancelled downloads.

    Skipping this for now.

  • Replace direct puts output with different output formatters for serial/concurrent download output.

  • Implement concurrent downloads for brew install, brew reinstall, brew upgrade etc. with a --concurrency flag. Replaced with HOMEBREW_DOWNLOAD_CONCURRENCY variable.

  • Implement global concurrent downloads with a public/documented/supported HOMEBREW_DOWNLOADS_CONCURRENCY=<int> opt-in variable.

  • Limit concurrent connections per host to avoid overloading smaller (i.e. non-CDN) web servers.

    Skipping this for now.

  • Enable concurrent downloads by default i.e. add HOMEBREW_NO_DOWNLOAD_CONCURRENCY and default HOMEBREW_DOWNLOAD_CONCURRENCY to auto.

Metadata

Metadata

Assignees

Labels

in progressMaintainers are working on this

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions