Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to Prefect runtime processor type. #2880

Open
jcfaracco opened this issue Aug 14, 2022 · 2 comments
Open

Add support to Prefect runtime processor type. #2880

jcfaracco opened this issue Aug 14, 2022 · 2 comments
Labels
kind:enhancement New feature or request platform: pipeline-BYO-runtime Request for support of a new runtime for Elyra pipelines (BYO = bring your own)

Comments

@jcfaracco
Copy link

Is your feature request related to a problem? Please describe.
As mentioned by #2258, it should be easy now to integrate new processor executor type into Elyra definition. This is just a tracker to support Prefect. Ideally, the implementation should be similar to Airflow considering the similarities between each other.

Describe the solution you'd like
A pipeline that supports Prefect runtime processor.

Describe alternatives you've considered
There is an alternative to create a generic pipeline with Prefect code, but it would be nice to have it built-in.

@jcfaracco jcfaracco added the kind:enhancement New feature or request label Aug 14, 2022
@ptitzler ptitzler added the platform: pipeline-BYO-runtime Request for support of a new runtime for Elyra pipelines (BYO = bring your own) label Aug 15, 2022
@jcfaracco
Copy link
Author

Is it possible to include runtimes as a plugin? So, we don't need to open requests like these. We could just maintain a separate elyra-<runtime_type> and if it is relevant, it can be incorporated in the future.

@akchinSTC
Copy link
Member

Is it possible to include runtimes as a plugin? So, we don't need to open requests like these. We could just maintain a separate elyra-<runtime_type> and if it is relevant, it can be incorporated in the future.

unfortunately we dont have the plumbing in place yet for a more modular way of including additional processor runtimes, this includes packaging (see in progress #2259).

Makes sense to keep this open as an umbrella issue for adding support for prefect in elyra, we would be happy to help answer any questions or issues that come up and look forward to this contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:enhancement New feature or request platform: pipeline-BYO-runtime Request for support of a new runtime for Elyra pipelines (BYO = bring your own)
Projects
None yet
Development

No branches or pull requests

3 participants