Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Infinite reconciliation loop when overlapping rules exist #432

Open
jamesstocktonj1 opened this issue Sep 28, 2024 · 0 comments
Open
Assignees
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@jamesstocktonj1
Copy link

Description

Consider a scenario with 4 wasmcloud instances with the following labels:

  • node 1 - region=us-east-1, cloud=real
  • node 2 - region=us-east-2, cloud=real
  • node 3 - region=us-west-1, cloud=real
  • node 4 - region=us-west-2, cloud=real

And the following spreadscale properties for a component:

type: spreadscaler
properties:
  instances: 12
  spread:
    - name: eastcoast
      requirements:
        region: us-east-1
      weight: 25
    - name: westcoast
      requirements:
        region: us-west-1
      weight: 25
    - name: realcloud
      requirements:
        cloud: real
      weight: 50

Problem

At first, wadm deploys 3 instances to node 1 and 3 instances to node 3. This satisfies the requirements eastcoast and westcoast but it also satisfies the requirement realcloud. Even though all the requirements are satisfied, the deployment is not successful because the total number of components deployed is 6 and not 12. Therefore Wadm continues the reconciliation process.

After this, in order to try and satisfy the number of instances it deploys 6 instances on either node 2 or node 4. This then satisfies the total number of instances but does not satisfy the weighting for realcloud as it is 100% and not 50%.

This cycle continues as wadm attempts to satisfy all conditions for deployment, resulting in a reconciliation loop with no exit.

TLDR

A spreadscaler config may have a set of requirements which cannot fully be satisfied so the deployment status remains as "Reconciling".

@jamesstocktonj1 jamesstocktonj1 changed the title [BUG] Infinite reconciling loop when overlapping rules exist [BUG] Infinite reconciliation loop when overlapping rules exist Sep 28, 2024
@brooksmtownsend brooksmtownsend added bug Something isn't working help wanted Extra attention is needed labels Oct 22, 2024
@ahmedtadde ahmedtadde self-assigned this Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
Status: No status
Development

No branches or pull requests

3 participants