Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chunk better #45

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Chunk better #45

wants to merge 1 commit into from

Conversation

treeowl
Copy link
Contributor

@treeowl treeowl commented Jun 21, 2018

parListChunk previously split a list up into chunks, applied
the given strategy to each chunk, and then put them all together
again. This led to two extra copies of the list.

We get very little benefit from actually splitting the list, because the
parallel computations need to traverse their part anyway; we can instead just
hand off the whole list and let them count out their chunk. We count each
chunk twice, but that shouldn't cost enough to matter.

Now that Eval has a MonadFix instance, we can avoid actually having
to put together lists at the end; instead, we pass each parallel
computation the (as-yet-uncomputed) result of calculating the rest
of the list.

`parListChunk` previously split a list up into chunks, applied
the given strategy to each chunk, and then put them all together
again. This led to two extra copies of the list.

We get very little benefit from actually splitting the list, because the
parallel computations need to traverse their part anyway; we can instead just
hand off the whole list and let them count out their chunk.  We count each
chunk twice, but that shouldn't cost enough to matter.

Now that `Eval` has a `MonadFix` instance, we can avoid actually having
to put together lists at the end; instead, we pass each parallel
computation the (as-yet-uncomputed) result of calculating the rest
of the list.
@simonmar
Copy link
Member

Great. Can we have some tests and benchmarks please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants