README.md

WPI's Workflow Resource Patterns and CWL

Note: Most of the WPI resource handling patterns concern the allocation of business tasks to human resources.

The CWL standards define a language to describe batch computation tasks and workflows made from these tasks. CWL purposely does not include non-computational tasks. Distribution of tasks in CWL is up to the implementing workflow engine to comply with any specified computational resource requirements (which are mandatory) or hints (which are optional).

While only 2 of the 43 of the WPI Resource Patterns have an analogue in the CWL standards, many of these other patterns are used by CWL compatible workflow management systems; either controlled by algorithmic choices or user/operator input.

Back to the list of WPI pattern categories

"Creation Patterns"

• Pattern 1 (Direct Distribution)

  No. There is no concept of a (computational) resource identity in the CWL standards. There is no method to specify that a given task should be execution on a specific computer.

• Pattern 2 (Role-Based Distribution)

  No.

• Pattern 3 (Deferred Distribution)

  No. The choice of up front scheduling, deferred scheduling, or hybrid scheduling is not a concept in CWL. The CWL standards do not specify a scheduling modality nor do they forbid any scheduling modality.

• Pattern 4 (Authorization)

  No.

• Pattern 5 (Separation of Duties)

  No.

• Pattern 6 (Case Handling)

  No. The CWL standards have no concept of a case.

• Pattern 7 (Retain Familiar)

  No. However a workflow engine is allowed to schedule/place jobs to take advantage of data locality as an optimization.

• Pattern 8 (Capability-Based Distribution)

  Yes, except there is no concept of "organisational model" in CWL. CWL CommandLineTools can define minimum needed data storage and memory needs, along with the minimum number of CPU cores for efficient computation. These ResourceRequirements influence task placement.

  CWL versions: all

• Pattern 9 (History-Based Distribution)

  No. Like Pattern 7 (Retain Faimiliar), this could be implemented by a workflow engine to optimize based upon the availability of intermediate results left over (or nearby) by a previous task.

• Pattern 10 (Organisational Distribution)

  No. CWL has no concept of an "organization" or any entity similar to that.

• Pattern 11 (Automatic Execution)

  By definition, all tasks in CWL are automatic as CWL has no concept of sending a task to a human.

  CWL versions: all

"Push Patterns"

• Pattern 12 (Distribution by Offer - Single Resource)

  No.

• Pattern 13 (Distribution by Offer - Multiple Resources)

  No.

• Pattern 14 (Distribution by Allocation - Single Resource)

  No.

• Pattern 15 (Random Allocation)

  No.

• Pattern 16 (Round Robin Allocation)

  No.

• Pattern 17 (Shortest Queue)

  No.

• Pattern 18 (Early Distribution)

  No.

• Pattern 19 (Distribution on Enablement)

  No.

• Pattern 20 (Late Distribution)

  No.

"Pull Patterns"

• Pattern 21 (Resource-Initiated Allocation)

  No.

• Pattern 22 (Resource-Initiated Execution - Allocated Work Item

  No.

• Pattern 23 (Resource-Initiated Execution - Offered Work Item)

  No.

• Pattern 24 (System-Determined Work Queue Content)

  No.

• Pattern 25 (Resource-Determined Work Queue Content)

  No.

• Pattern 26 (Selection Autonomy)

  No.

• Pattern 21 (Resource-Initiated Allocation)

  No.

"Detour Patterns"

• Pattern 27 (Delegation)

  No.

• Pattern 28 (Escalation)

  No.

• Pattern 29 (Deallocation)

  No.

• Pattern 30 (Stateful Reallocation)

  No.

• Pattern 31 (Stateless Reallocation)

  No.

• Pattern 32 (Suspension/Resumption)

  No.

• Pattern 33 (Skip)

  No.

• Pattern 34 (Redo)

  No. There is no method in CWL for a task to specify that some other already completed task must be redone.

  However, if a CommandLineTool exits with an return code indicated in the specified temporaryFailCodes list, then the task is available for retrying again. The CWL standards do not specify how many retries (if any) should be attempted.

• Pattern 35 (Pre-Do)

  No. In CWL it is assumed that once a task has all of its inputs it is made available for execution.

"Auto-Start Patterns"

• Pattern 36 (Commencement on Creation)

  No.

• Pattern 37 (Commencement on Allocation)

  No.

• Pattern 38 (Piled Execution)

  No. There is no specific construct in CWL to force pipelined execution, however the presence of streamable: true on matching inputs/outputs of adjacent tasks is a strong hint for those system who wish to do so.

• Pattern 39 (Chained Execution)

  No.

"Visibility Patterns"

• Pattern 40 (Configurable Unallocated Work Item Visibility)

  No.

• Pattern 41 (Configurable Allocated Work Item Visibility)

  No.

"Multiple Resource Patterns"

• Pattern 42 (Simultaneous Execution)

  No, there is no explicit CWL construct for this. It is assumed that parallel execution, when possible, is wanted by the user. All known CWL implementations support parallel execution, and almost all of them do so by default.

• Pattern 43 (Additional Resources)

  No.