Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Workers with single decision poller with sticky tasklist feature enabled will stop processing #1369

Open
shijiesheng opened this issue Oct 8, 2024 · 2 comments
Open

Workers with single decision poller with sticky tasklist feature enabled will stop processing #1369

shijiesheng opened this issue Oct 8, 2024 · 2 comments
Labels
bug

Comments

@shijiesheng
Copy link
Member

Describe the bug
Decision worker will stop processing if decision poller count is 1 and sticky tasklist is enabled

To Reproduce
MaxConcurrentDecisionTaskPollers is set to 1 (default 2)
Or MinConcurrentDecisionTaskPollers is set to 1 and Poller Auto Scaler is enabled

Expected behavior
Workflow workers should process decision tasks
@shijiesheng
Copy link
Member Author

Root Cause
Current implementation is:
X number of decision goroutines are started
Each goroutine will either call sticky tasklist or normal tasklist based on metrics
When X = 1, the condition will always be true. This causes decision polling to stop for normal tasklist

Temporal keeps the same logic https://github.com/temporalio/sdk-go/blob/cdd30702f1ba479b57111763095e24a25a04ea53/internal/internal_task_pollers.go#L781

@shijiesheng
Copy link
Member Author

We decided to add enforcement to have at least 2 decision pollers to avoid breaking existing behaviors

@shijiesheng shijiesheng mentioned this issue Oct 8, 2024
Merged
@shijiesheng shijiesheng reopened this Oct 8, 2024
shijiesheng added a commit that referenced this issue Oct 10, 2024
@shijiesheng
added worker.NewV2 with validation on decision poller count (#1370)
89676d3 
What changed?

added a worker.NewV2 interface that will return error and a deprecation message on New api.
added validation on WorkerOptions
-- decision poller should be larger than 1 if value is set (not zero) and sticky execution is enabled.
Why?
#1369

How did you test it?
Unit Test

Potential risks
Low
@ibarrajo ibarrajo added the bug label Nov 4, 2024
mrombout pushed a commit to softsense/cadence-client that referenced this issue Feb 26, 2025
@shijiesheng @mrombout
added worker.NewV2 with validation on decision poller count (cadence-…
022a993 
…workflow#1370)

What changed?

added a worker.NewV2 interface that will return error and a deprecation message on New api.
added validation on WorkerOptions
-- decision poller should be larger than 1 if value is set (not zero) and sticky execution is enabled.
Why?
cadence-workflow#1369

How did you test it?
Unit Test

Potential risks
Low
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ibarrajo @shijiesheng