Skip to content

Use label selectors and upgrade master Nodes first #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 6, 2025

Conversation

jimmykarily
Copy link
Contributor

@jimmykarily jimmykarily commented Jun 5, 2025

We should be targeting Nodes in a more "Kubernetes native" way, using label selectors. The Kairos node labeler will also make more sense now. We automatically label Kairos nodes. The label can be used to target only Kairos nodes for upgrades. For now this targeting is left to the user but we can make it automatic in the future is we so desire.

This PR also:

  • Sorts nodes with masters first. This makes the canary upgrades, upgrade master first which means we avoid deadlock situations in which a worker node running the operator gets rebooted in to a non-compatible (usually older) kubernetes version. This results in the worker not connecting to the cluster, and the operator not starting again. Thus, the rest of the Nodes never get an Upgrade Job create, stopping the whole process. The workaround would be to kill the operator Pod and let it be schefuled on a Node that is still schedulable. But doing masters first, ensures that there will be a compatible master to connect to when the operator Node comes back up.
  • Refactors the code that handles the NodeOp status. Now there is a method to update the "Phase" and a method to update the reboot status.

Part of: kairos-io/kairos#769

@jimmykarily jimmykarily marked this pull request as ready for review June 5, 2025 11:12
and use unique names in all tests to avoid leaking

Signed-off-by: Dimitris Karakasilis <[email protected]>
Split the job completion logic and the reboot completion logic into 2
different methods. Then handle uncordoning in the caller.

Also fix tests.

Signed-off-by: Dimitris Karakasilis <[email protected]>
@jimmykarily
Copy link
Contributor Author

I need to split this PR in at least 2. The description only describes the first commit.

@jimmykarily jimmykarily marked this pull request as draft June 5, 2025 14:55
@jimmykarily jimmykarily marked this pull request as ready for review June 6, 2025 05:37
@jimmykarily
Copy link
Contributor Author

I need to split this PR in at least 2. The description only describes the first commit.

updated the description instead of splitting (because the PR was already approved)

@jimmykarily jimmykarily changed the title Switch from Node names to a label selector Use label selectors and upgrade master Nodes first Jun 6, 2025
@jimmykarily jimmykarily moved this to Under review 🔍 in 🧙Issue tracking board Jun 6, 2025
@jimmykarily jimmykarily self-assigned this Jun 6, 2025
@jimmykarily jimmykarily merged commit 3d49081 into main Jun 6, 2025
7 checks passed
@github-project-automation github-project-automation bot moved this from Under review 🔍 to Done ✅ in 🧙Issue tracking board Jun 6, 2025
@jimmykarily jimmykarily deleted the label-selectors branch June 6, 2025 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants