Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement local shuffle optimization for broadcast relations #549

Open
senderista opened this issue Jan 6, 2017 · 0 comments
Open

Implement local shuffle optimization for broadcast relations #549

senderista opened this issue Jan 6, 2017 · 0 comments
Assignees

Comments

@senderista
Copy link
Contributor

@stechu had a simple and beautiful idea: for broadcast inputs to a hash-partitioned shuffle, we can just filter them in-place to retain only tuples satisfying the hash partition condition for that worker. The same would apply to round-robin shuffles, except we would partition the tuple count on uniform boundaries. This might require support on the MyriaX side.

@senderista senderista self-assigned this Jan 6, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant