Add concurrency policy #3267

thomasjhuang · 2025-06-06T21:38:18Z

Tracking issue

This is associated to flyte change flyteorg/flyte#6475, and RFC flyteorg/flyte#5659

Why are the changes needed?

These are the necessary changes needed to implement the ConcurrencyPolicy within flytekit. Namely, it will allow users to manage concurrency control via LaunchPlan.create() or LaunchPlan.get_or_create()

How was this patch tested?

Unit tests are added, local testing is also done.

Screenshots

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Docs link

Summary by Bito

This pull request introduces a new concurrency policy feature in the Flytekit library, enhancing concurrency control through LaunchPlan methods. It updates existing models and launch plan parameters to incorporate these settings, improving overall functionality. Comprehensive unit tests have been added to ensure reliability and system integrity.

welcome · 2025-06-06T21:38:20Z

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
Sign off your commits (Reference: DCO Guide).

Sovietaced · 2025-07-13T17:37:20Z

@thomasjhuang Can you sign off on these commits? And can someone else let the CI run?

thomasjhuang · 2025-07-22T21:40:19Z

@thomasjhuang Can you sign off on these commits? And can someone else let the CI run?

Sorry about that, added the signoff

popojk · 2025-08-05T07:01:10Z

flytekit/models/concurrency.py

+
+
+class ConcurrencyLimitBehavior(object):
+    SKIP = _launch_plan_idl.ConcurrencyLimitBehavior.SKIP


I encountered some errors in my local execution, According to the latest flyteidl, should we use SKIP = _launch_plan_idl.ConcurrencyLimitBehavior.CONCURRENCY_LIMIT_BEHAVIOR_SKIP here?

Yea I need to update this according to the merged PR in flyte. Thanks for the callout

kumare3 · 2025-08-25T17:41:04Z

LGTM

flytekit/models/concurrency.py

thomasjhuang · 2025-09-22T04:51:17Z

@kumare3 @davidmirror-ops These test failures are a little odd to me, they seem transient? would a rerun suffice here?

kumare3 · 2025-09-22T14:02:44Z

Hmm weird how did the event loop vanish

machichima · 2025-09-23T06:21:36Z

I fixed the failed test in https://github.com/flyteorg/flytekit/pull/3265/files#r2354426173
@thomasjhuang could you please merge/rebase master and see if tests can pass? Thank you!

thomasjhuang · 2025-09-24T20:36:38Z

I fixed the failed test in https://github.com/flyteorg/flytekit/pull/3265/files#r2354426173 @thomasjhuang could you please merge/rebase master and see if tests can pass? Thank you!

Ah got it, thanks I've rebased and pushed again.

machichima · 2025-09-25T09:57:34Z

tests/flytekit/unit/models/test_concurrency.py

+
+
+def test_concurrency_limit_behavior():
+    assert ConcurrencyLimitBehavior.SKIP == _launch_plan_idl.ConcurrencyLimitBehavior.SKIP


Suggested change

assert ConcurrencyLimitBehavior.SKIP == _launch_plan_idl.ConcurrencyLimitBehavior.SKIP

assert ConcurrencyLimitBehavior.SKIP == _launch_plan_idl.CONCURRENCY_LIMIT_BEHAVIOR_SKIP

@thomasjhuang
All _launch_plan_idl.ConcurrencyLimitBehavior.SKIP below should be changed to _launch_plan_idl.CONCURRENCY_LIMIT_BEHAVIOR_SKIP. This can solve the error AttributeError: Enum ConcurrencyLimitBehavior has no value defined for name 'SKIP'

Updated, I noticed the issue yesterday as well. Thanks for the callout!

thomasjhuang · 2025-09-26T17:39:56Z

@kumare3 @davidmirror-ops @popojk @machichima Looks like all checks have passed now! Thanks for the help :)

davidmirror-ops · 2025-09-26T18:53:53Z

Thank you @thomasjhuang!

machichima · 2025-09-29T00:15:41Z

flytekit/core/launch_plan.py

        :param auto_activate: If set to True, the launch plan will be activated automatically on registration.
+        :param concurrency: Defines execution concurrency limits and policy when limit is reached
         Default is False.


Is the docstring here being cut-off?

No, but I don't see the issue with this line?

Suggested change

:param auto_activate: If set to True, the launch plan will be activated automatically on registration.

:param concurrency: Defines execution concurrency limits and policy when limit is reached

Default is False.

:param auto_activate: If set to True, the launch plan will be activated automatically on registration.

Default is False.

:param concurrency: Defines execution concurrency limits and policy when limit is reached.

Oh I mean "Default is False" is for auto_activate?

I see! fixed it, thanks!

tests/flytekit/unit/models/test_concurrency.py

Signed-off-by: thomasjhuang <[email protected]>

Co-authored-by: Nary Yeh <[email protected]> Signed-off-by: thomasjhuang <[email protected]>

Signed-off-by: thomasjhuang <[email protected]>

machichima

LGTM. Thanks!!

welcome · 2025-10-03T00:14:09Z

Congrats on merging your first pull request! 🎉

* Add concurrency policy Signed-off-by: thomasjhuang <[email protected]> * Update to latest idl Signed-off-by: thomasjhuang <[email protected]> * Reformat with ruff Signed-off-by: thomasjhuang <[email protected]> * Change reference Signed-off-by: thomasjhuang <[email protected]> * Update test Signed-off-by: thomasjhuang <[email protected]> * Update tests/flytekit/unit/models/test_concurrency.py Co-authored-by: Nary Yeh <[email protected]> Signed-off-by: thomasjhuang <[email protected]> * Update comment Signed-off-by: thomasjhuang <[email protected]> * Fix lint Signed-off-by: thomasjhuang <[email protected]> --------- Signed-off-by: thomasjhuang <[email protected]> Co-authored-by: Nary Yeh <[email protected]> Signed-off-by: Atharva <[email protected]>

thomasjhuang requested review from cosmicBboy, davidmirror-ops, eapolinario, kumare3, pingsutw, samhita-alla and wild-endeavor as code owners June 6, 2025 21:38

thomasjhuang mentioned this pull request Jun 6, 2025

Add workflow concurrency control flyteorg/flyte#6475

Merged

3 tasks

thomasjhuang force-pushed the thomasjhuang/concurrency branch from 167ea31 to 57e7827 Compare July 22, 2025 21:39

popojk reviewed Aug 5, 2025

View reviewed changes

thomasjhuang force-pushed the thomasjhuang/concurrency branch from 168a830 to 6b353ee Compare August 15, 2025 22:46

thomasjhuang mentioned this pull request Aug 19, 2025

Add concurrency control docs to launch plan section unionai/unionai-docs#417

Merged

kumare3 previously approved these changes Aug 25, 2025

View reviewed changes

thomasjhuang dismissed kumare3’s stale review via 77059fa September 8, 2025 18:10

thomasjhuang force-pushed the thomasjhuang/concurrency branch from 77059fa to d9dce8e Compare September 8, 2025 18:12

davidmirror-ops requested review from machichima and popojk September 9, 2025 10:42

davidmirror-ops reviewed Sep 9, 2025

View reviewed changes

flytekit/models/concurrency.py Show resolved Hide resolved

kumare3 previously approved these changes Sep 11, 2025

View reviewed changes

thomasjhuang dismissed kumare3’s stale review via 4da0014 September 16, 2025 19:11

thomasjhuang force-pushed the thomasjhuang/concurrency branch from 4da0014 to 012276c Compare September 17, 2025 03:00

thomasjhuang force-pushed the thomasjhuang/concurrency branch from 012276c to 55bf831 Compare September 24, 2025 20:35

machichima reviewed Sep 25, 2025

View reviewed changes

thomasjhuang force-pushed the thomasjhuang/concurrency branch from a029dab to a0f6004 Compare September 25, 2025 20:47

davidmirror-ops enabled auto-merge (squash) September 26, 2025 18:53

machichima reviewed Sep 29, 2025

View reviewed changes

auto-merge was automatically disabled September 29, 2025 18:03
Head branch was pushed to by a user without write access

thomasjhuang force-pushed the thomasjhuang/concurrency branch from 71f92e4 to 68d189a Compare September 30, 2025 18:46

thomasjhuang and others added 6 commits October 2, 2025 10:29

Add concurrency policy

57c6853

Signed-off-by: thomasjhuang <[email protected]>

Update to latest idl

b2c1b4f

Signed-off-by: thomasjhuang <[email protected]>

Reformat with ruff

e99cf57

Signed-off-by: thomasjhuang <[email protected]>

Change reference

bf2e179

Signed-off-by: thomasjhuang <[email protected]>

Update test

aef83a8

Signed-off-by: thomasjhuang <[email protected]>

Update tests/flytekit/unit/models/test_concurrency.py

e1ba971

Co-authored-by: Nary Yeh <[email protected]> Signed-off-by: thomasjhuang <[email protected]>

thomasjhuang force-pushed the thomasjhuang/concurrency branch from 68d189a to e1ba971 Compare October 2, 2025 17:30

thomasjhuang added 2 commits October 2, 2025 14:08

Update comment

694ad1b

Signed-off-by: thomasjhuang <[email protected]>

Fix lint

d618d21

Signed-off-by: thomasjhuang <[email protected]>

machichima approved these changes Oct 2, 2025

View reviewed changes

kumare3 approved these changes Oct 2, 2025

View reviewed changes

machichima merged commit b4050b6 into flyteorg:master Oct 3, 2025
115 checks passed



		class ConcurrencyLimitBehavior(object):
		SKIP = _launch_plan_idl.ConcurrencyLimitBehavior.SKIP



		def test_concurrency_limit_behavior():
		assert ConcurrencyLimitBehavior.SKIP == _launch_plan_idl.ConcurrencyLimitBehavior.SKIP

Add concurrency policy #3267

Add concurrency policy #3267

Uh oh!

Conversation

thomasjhuang commented Jun 6, 2025 • edited by flyte-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tracking issue

Why are the changes needed?

How was this patch tested?

Screenshots

Check all the applicable boxes

Related PRs

Docs link

Summary by Bito

Uh oh!

welcome bot commented Jun 6, 2025

Uh oh!

Sovietaced commented Jul 13, 2025

Uh oh!

thomasjhuang commented Jul 22, 2025

Uh oh!

popojk Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

thomasjhuang Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

kumare3 commented Aug 25, 2025

Uh oh!

Uh oh!

thomasjhuang commented Sep 22, 2025

Uh oh!

kumare3 commented Sep 22, 2025

Uh oh!

machichima commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thomasjhuang commented Sep 24, 2025

Uh oh!

machichima Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thomasjhuang Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

thomasjhuang commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidmirror-ops commented Sep 26, 2025

Uh oh!

machichima Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

thomasjhuang Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

machichima Oct 2, 2025

Choose a reason for hiding this comment

Uh oh!

thomasjhuang Oct 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

machichima left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

welcome bot commented Oct 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

thomasjhuang commented Jun 6, 2025 •

edited by flyte-bot

Loading

machichima commented Sep 23, 2025 •

edited

Loading

machichima Sep 25, 2025 •

edited

Loading

thomasjhuang commented Sep 26, 2025 •

edited

Loading