Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stateful apply rule for MyriaX #535

Open
billhowe opened this issue Aug 18, 2016 · 1 comment
Open

Stateful apply rule for MyriaX #535

billhowe opened this issue Aug 18, 2016 · 1 comment
Assignees

Comments

@billhowe
Copy link
Contributor

Need to stream all results from a partitioned dataset through a single worker for stateful apply to have the correct semantics.

OR notice that the entire dataset is already on one worker

AND EVENTUALLY inspect the update function to determine that it can be computed algebraically in parallel as a hierarchy. (This one is hardest, but the best)

@shrjain shrjain self-assigned this Aug 18, 2016
@senderista
Copy link
Contributor

This still doesn't address the ordering issue, right? Do we need to require an ordering on the input to StatefulApply (which would require global ORDER BY support), or do we just assume that the update function is commutative?

Also see #158

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants