Flyte 2 WIP by EngHabu · Pull Request #6583 · flyteorg/flyte

EngHabu · 2025-08-22T06:38:23Z

Add flyteidl and minimal publishing workflows

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Please add one or more of the following labels to categorize your PR:

added: For new features.
changed: For changes in existing functionality.
deprecated: For soon-to-be-removed features.
removed: For features being removed.
fixed: For any bug fixed.
security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Docs link

main
- Flyte 2 WIP #6583 👈

* Add flyteidl and minimal publishing workflows Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * pretend version for buf python Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

flyte-bot · 2025-08-22T06:38:58Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

* python version fix and use OIDC to authenticate crates Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Fix python versioning Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Set version before setting up env Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * echo version Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Write to GITHUB_ENV Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Use env: Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * buf generate Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

flyte-bot · 2025-08-22T18:38:23Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

flyte-bot · 2025-08-25T20:39:37Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

* Mark go package as /v2 to be able to import both at the same time Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Upgrade to go 1.24 Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Rename python, rust and npm to flyteidl2 Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

flyte-bot · 2025-08-26T01:51:47Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

…6610) * trying idl2 Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Update trace def Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor: rename idl2 to flyteidl2 Signed-off-by: machichima <nary12321@gmail.com> * generate -> buf Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor: rename idl2 to flyteidl2 Signed-off-by: machichima <nary12321@gmail.com> * build: re-gen Signed-off-by: machichima <nary12321@gmail.com> * fix: exclude python venv Signed-off-by: machichima <nary12321@gmail.com> * Add translator service Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Regen Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * buf gen Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * remove node execution service proto Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Breakup task service into a separate package Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Help strings for make file targets Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Update TaskLog with read bool Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Trying the updated protovalidate plugin Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * protovalidate with semver Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * try claude :shrug Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * add buf Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * update Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * update deps Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * build: remove generate validate for python Signed-off-by: machichima <nary12321@gmail.com> * build: use managed mode on buf.gen.yaml Signed-off-by: machichima <nary12321@gmail.com> * Update validate go plugin Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Update imagebuild Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Fix oneof buf lint Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Add deps to generated python code Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Be explicit about the index Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * updated idl Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Catch up to main Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Generate mocks for workflow package Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * generate mocks for other packages Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * buf improvements Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * updates Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * feat: generate mock for common Signed-off-by: machichima <nary12321@gmail.com> * use make gen in GH Actions Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Do not setup python env in generation CI Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * setup uv Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * set working directory Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Try to reference files by path Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Explicitly move .venv to root Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * try exclude in buf.yaml Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * set UV_PROJECT_ENVIRONMENT Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * download tooling Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * add which mockery Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> Signed-off-by: machichima <nary12321@gmail.com> Co-authored-by: machichima <nary12321@gmail.com>

flyte-bot · 2025-09-22T20:23:04Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at eduardo@union.ai.

* Add Trigger APIs Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * update triggers Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * wip Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * refactor Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

bito-code-review · 2025-09-30T18:23:19Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

* Fix cargo build and add CI check Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * lint lib.rs Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Update Readme and contributing guides Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * signed? Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix build target Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix build-crate target Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix included ts files in npm Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

bito-code-review · 2025-09-30T22:29:13Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

bito-code-review · 2025-10-03T18:55:43Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

* Add device class to `GpuAccelerator` Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * make gen Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * Add HABANA_GAUDI as supported device class Signed-off-by: Jeev B <jeevb@users.noreply.github.com> --------- Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

bito-code-review · 2025-10-04T00:01:40Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

bito-code-review · 2025-10-04T04:21:41Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

bito-code-review · 2025-10-07T18:11:43Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

bito-code-review · 2025-10-08T19:51:43Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

* Clean up triggers api Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai> * make gen Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai> --------- Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

bito-code-review · 2025-10-09T18:11:16Z

Bito Automatic Review Skipped - Branch Excluded

Bito didn't auto-review because the source or target branch is excluded from automatic reviews.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the branch exclusion settings here, or contact your Bito workspace admin at haytham@union.ai.

* Fix vet issues Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * Add git-town tracking Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * gen Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> * fix: use GetRunAs with nil checker Signed-off-by: machichima <nary12321@gmail.com> * fix: remove Cargo.lock Signed-off-by: machichima <nary12321@gmail.com> * fix: local fix cherry pick state etcd Signed-off-by: machichima <nary12321@gmail.com> --------- Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com> Signed-off-by: machichima <nary12321@gmail.com> Co-authored-by: machichima <nary12321@gmail.com>

Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Kevin Su <pingsutw@apache.org>

Read error outputs from the output reader and translate them into appropriate phase transitions (retryable or permanent failure) before writing outputs, preventing silent error swallowing. Signed-off-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org>

Signed-off-by: Kevin Su <pingsutw@apache.org>

Add Abort APIs (#6590)

82a41b6

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Vendor in python buf dependencies (#6651)

1993238

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

EngHabu added 5 commits October 3, 2025 21:20

wip

7bb0db0

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

queue service implementation

075f324

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Support sqllite

987d536

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Runs service

aa24751

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Connect RunService to QueueService

3ef9544

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>

Triggers API adjustments (#6655)

83acecf

Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

RunService.CreateRun from_automation flags (#6653)

cda285a

Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

Cleanup triggers api (#6666)

a480286

* Clean up triggers api Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai> * make gen Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai> --------- Signed-off-by: Iaroslav Ciupin <iaroslav@union.ai>

[V2] Migrate action service (#6949)

6a55582

Signed-off-by: machichima <nary12321@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Kevin Su <pingsutw@apache.org>

github-actions bot mentioned this pull request Feb 28, 2026

feat(flyteidl2): add image_build_url field to TaskMetadata proto #6951

Open

3 tasks

pingsutw added 2 commits February 27, 2026 17:40

Add source_dir to UVProject proto (#6939)

cc0535a

Signed-off-by: Kevin Su <pingsutw@apache.org>

Conversation

EngHabu commented Aug 22, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Setup process

Screenshots

Check all the applicable boxes

Related PRs

Docs link

Uh oh!

flyte-bot commented Aug 22, 2025

Uh oh!

flyte-bot commented Aug 22, 2025

Uh oh!

flyte-bot commented Aug 25, 2025

Uh oh!

flyte-bot commented Aug 26, 2025

Uh oh!

flyte-bot commented Sep 22, 2025

Uh oh!

bito-code-review bot commented Sep 30, 2025

Uh oh!

bito-code-review bot commented Sep 30, 2025

Uh oh!

bito-code-review bot commented Oct 3, 2025

Uh oh!

bito-code-review bot commented Oct 4, 2025

Uh oh!

bito-code-review bot commented Oct 4, 2025

Uh oh!

bito-code-review bot commented Oct 7, 2025

Uh oh!

bito-code-review bot commented Oct 8, 2025

Uh oh!

bito-code-review bot commented Oct 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

EngHabu commented Aug 22, 2025 •

edited by github-actions bot

Loading