ADBDEV-8673: Add support for ShareInputScans inside InitPlans #108
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously ShareInputScans could not be used inside InitPlans due to synchronization issues. This happened because InitPlans are executed sequentially: they are separate queries dispatched to QEs, and each query has to complete before the next one is started.
This conflicts with ShareInputScan execution (in cross-slice case), which assumes that producer and consumer slices are both available at the same time, so producer must notify consumers through shared when it's done preparing the tuplestore, and the consumers must notify the producer when they're done reading it, so it's okay to producer to delete the tuplestore.
Synchronization is actually rather hard, since InitPlans can contain several ShareInputScans, which must be synchronized normally. So the producer must still wait for all the consumers in the same InitPlan, but not for consumers in other InitPlans, and not for consumers in the main plan. Similarly, only the consumers in the same InitPlan must wait for the producer. For that we must somehow determine
Cleanup is harder to deal with, since producer has to clean up the tuplestore after the consumers are done, which is after producer's InitPlan is completed and its transaction commited. This means we must store the tuplestore in some separate global place that can be kept alive even after the transaction commit, and be cleaned up after the whole command finishes (as determined by gp_command_id).