Resource-limited flow graph nodes

[akukanov]

This discussion is to collect opinions/ideas regarding further evolution of the RFC #1677.

[akukanov]

Some additional context that is relevant for further work on the idea.

While the original motivation was about preventing concurrent access to non-safe program components by multiple nodes, the direction it seemingly moves toward is about generalized resource control. That very similar to other FG use cases and techniques that we know for a while, including:
• The “dining philosophers” example (https://github.com/uxlfoundation/oneTBB/tree/master/examples/graph/dining_philosophers); in a sense, it can be seen as a “distilled” version of the Fermilab usage pattern, in which several functional nodes must acquire the same resource before executing and must return it to the shared pool afterwards.
• The token based approach (https://uxlfoundation.github.io/oneTBB/main/tbb_userguide/create_token_based_system.html) where token pools are potentially “shared” by multiple nodes.
• Patterns using the limiter_node (https://uxlfoundation.github.io/oneTBB/main/tbb_userguide/use_limiter_node.html), if, besides a counter, the node would also have a buffer of resource handles.

We may want to somehow simplify expression of such patterns (including examples in #1677 with the flow graph. So, we need to look at their similarities and differences, and figure out which (if any) new node types should be added and/or how the existing node types can be adjusted for the purpose.

[vossmjp]

One aspect that has been raised by #1677 is that we need a better way to combine token or resource use and concurrency limits. It may be that a serial node also needs a resource. Existing patterns decouple the token/resource acquisition from concurrency limits. For example, a reserving join node might get a token and data and forward to a serial function node that is currently occupied. This leads to situations where a token/resource is acquired for a node but then the resource cannot be immediately used, causing unnecessary idleness in the graph. Two possible approaches are to bring the reservation consumption closer to the node, so it is only acquired when there is also sufficient concurrency headroom available, or to make concurrency limits a resource/token that can be consumed by a join node (or other node type) outside of the concurrency-limited node.