Skip to content

Add scale sets #376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 46 commits into from
May 6, 2025
Merged

Add scale sets #376

merged 46 commits into from
May 6, 2025

Conversation

gabriel-samfira
Copy link
Member

@gabriel-samfira gabriel-samfira commented Apr 16, 2025

This PR adds scale set support to GARM.

Adding a new scale set is identical to adding a pool:

garm-cli scaleset add \
    --repo  30c6c4c7-e020-4203-b4f5-80f857b93238 \
    --provider-name yourProviderName \
    --image ubuntu:22.04 \
    --name test-scale-set \
    --flavor default \
    --enabled=true

Scalesets have the same commands that pools have:

ubuntu@garm:~$ garm-cli scaleset --help
Query information or perform operations on scale sets.

Usage:
  garm-cli scaleset [command]

Available Commands:
  add         Add scale set
  delete      Delete scale set by ID
  list        List scale sets
  show        Show details for a scale set
  update      Update one scale set

Flags:
  -h, --help   help for scaleset

Global Flags:
      --debug           Enable debug on all API calls
      --format string   Output format (table, json) (default "table")

Use "garm-cli scaleset [command] --help" for more information about a command.

Runners will appear along side the pool runners in:

garm-cli runner ls -a

@gabriel-samfira gabriel-samfira force-pushed the scalesets branch 7 times, most recently from 0d1c0a1 to f099268 Compare May 1, 2025 19:30
@gabriel-samfira gabriel-samfira marked this pull request as ready for review May 1, 2025 19:46
@gabriel-samfira
Copy link
Member Author

This still needs a lot of tests, but marking it as ready for review, as it should be fairly functional if anyone feels brave enough to test it.

@gabriel-samfira gabriel-samfira changed the title [WiP] Add scale sets Add scale sets May 1, 2025
This change moves the github client to a subpackage in utils
and adds the scaleset github client code.

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
The locking logic was added to its own package as it may need to be used
by other parts of the code.

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
This adds the workers needed to start listening for scale set messages.
There is no handling of messages yet.

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
* Close response body in scaleset client
* Wait for message listener loop to exit before attempting restart
* Add LastMessageID field to scaleset model and function to update it

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
It seems that Status is a string when you create a session, but a number
when you refresh it (?).

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
We will need to run various checks against the runners that are managed
by a scale set. The runners are updated by the DB watcher, so we should
always have an up to date view of their state. We can then confidently
monitor them without needing to query the DB.

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Runners now get created and cleaned up in scale sets.

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Fixes provider not spawning runners for newly added scale set

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Adds a periodic cleanup function that cross checks runners between github,
the provider and the GARM database. If an inconsistency is found, GARM will
attempt to fix it.

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
You can have multiple scale sets with the same name, as long as
they live in different runner groups.

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
* Split the main function into a couple of more functions
* Add credentials, entity, pool and scaleset cache
* add credentials worker that updates the cache

Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
Signed-off-by: Gabriel Adrian Samfira <[email protected]>
@gabriel-samfira
Copy link
Member Author

This has grown quite a bit. Merging this as is and will propose new PRs for additional tests and docs updates.

@gabriel-samfira gabriel-samfira merged commit e49b35d into cloudbase:main May 6, 2025
4 checks passed
@gabriel-samfira gabriel-samfira deleted the scalesets branch May 6, 2025 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant