-
Notifications
You must be signed in to change notification settings - Fork 17
Open
Labels
Description
We currently support (at least) two flags that control how workflows are scheduled:
--use_dynamic_scheduler, which is aboolindicating whether the scheduler invokesDynamicScheduleDAGorScheduleDAG.--use_heuristic, which is aboolindicating whether to use exhaustive search or the dynamic programming algorithm to find the system assignments (this manifests itself by callingSchedulerDynamic::ComputeOptimalorSchedulerDynamic::ComputeHeuristic).
IIRC, the difference between a dynamic and a non-dynamic scheduling call is whether Musketeer re-assesses its decisions after each job completion (@ICGog -- correct?).
The naming of things is a bit misleading here, for two reasons:
- The scheduler class is called
SchedulerDynamic, but the--use_dynamic_schedulerflag controls which method in this class gets called, rather than whether aSchedulerDynamicor something else is used. - Since our heuristic scheduler is based on dynamic programming, it is easy to assume that
--use_dynamic_schedulerrefers to the heuristic, which it does not.
Proposal: let's rename the flags, so that they're more intuitive.
--use_dynamic_schedulerbecomes--continuously_reschedule--use_heuristicbecomes a--scheduling_algorithmflag that can take three values: "automatic" (use heuristic if >18 operators, optimal otherwise), "heuristic" and "optimal".
This came up while investigating bugs reported by @n1v0lg.