Add remote workflow execution support to pyflyte run #3338
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tracking issue
N/A
Why are the changes needed?
Currently, the
pyflyte run
command supports executing remote tasks (remote-task
) and remote launch plans (remote-launchplan
), but lacks support for executing remote workflows directly. This creates an inconsistent user experience and limits flexibility when users want to execute workflows that are already registered in a Flyte backend.What changes were proposed in this pull request?
This PR adds support for executing remote workflows via the
pyflyte run remote-workflow
command, bringing feature parity with the existing remote task and launch plan execution capabilities.Changes include:
WORKFLOW_LAUNCHER
constant toDynamicEntityLaunchCommand
class_fetch_entity()
method to handle the workflow launcher type by callingfetch_workflow()
on the remote instancename:version
syntaxget_command()
inRemoteEntityGroup
to create aDynamicEntityLaunchCommand
with the workflow launcher when the command isremote-workflow
test_remote_workflow
andtest_remote_workflow_with_version
) to verify the functionalityHow was this patch tested?
Added two new unit tests in
tests/flytekit/unit/cli/pyflyte/test_run.py
:test_remote_workflow
: Tests basic remote workflow execution with parameterstest_remote_workflow_with_version
: Tests version-specific workflow execution using thename:version
syntaxBoth tests:
FlyteRemote
andrun_remote
to verify correct behaviorCliRunner
to simulate command-line executionfetch_workflow()
is called with the correct argumentsAll tests pass successfully:
Setup process
N/A
Screenshots
N/A
Check all the applicable boxes
Related PRs
N/A
Docs link
N/A
Summary by Bito
This pull request enhances the pyflyte run command by enabling the execution of remote workflows, introducing a WORKFLOW_LAUNCHER constant, and modifying the _fetch_entity method for workflow compatibility. It also supports version-specific workflows and includes comprehensive unit tests for reliability.