Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create jobs to separate AWS environments from build environments #481

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Create jobs to separate AWS environments from build environments #481

wants to merge 6 commits into from

Conversation

klutchell
Copy link
Contributor

@klutchell klutchell commented Jan 2, 2025 •
edited
Loading

This allows more granular control over which environments are used for which steps, and avoids requiring a single environment to define how to build, sign, and publish.

Initially this workflow will still fallback to the existing monolith environments and the behaviour should be unchanged.

See: https://balena.fibery.io/Work/Improvement/Update-yocto-scripts-workflow-with-separate-jobs-for-AWS-and-signing-environments-2471

image

Here are the workflows still pinned to master, generally meaning their builds are broken either way or Renovate would have pinned them to the current yocto-scripts submodule commit.
https://github.com/search?type=code&q=yocto-build-deploy.yml%40master+NOT+repo%3Abalena-os%2Fmeta-balena

@klutchell klutchell temporarily deployed to balena-staging.com January 2, 2025 19:21 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 2, 2025 19:21 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 2, 2025 19:58 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 2, 2025 19:58 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 2, 2025 19:59 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 2, 2025 20:31 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 2, 2025 20:31 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 2, 2025 20:31 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 3, 2025 16:25 — with GitHub Actions Inactive
github-advanced-security[bot]
github-advanced-security bot found potential problems Jan 3, 2025
View reviewed changes
.github/workflows/yocto-build-deploy.yml Fixed Show fixed Hide fixed
.github/workflows/yocto-build-deploy.yml Dismissed Show dismissed Hide dismissed
.github/workflows/yocto-build-deploy.yml Fixed Show fixed Hide fixed
@klutchell klutchell temporarily deployed to balena-staging.com January 3, 2025 16:26 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 3, 2025 16:26 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 3, 2025 17:01 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 3, 2025 17:01 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 3, 2025 17:01 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 3, 2025 17:32 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 3, 2025 17:33 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 3, 2025 17:33 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 3, 2025 18:37 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 3, 2025 18:37 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-cloud.com January 3, 2025 18:37 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 3, 2025 19:30 — with GitHub Actions Inactive
github-advanced-security[bot]
github-advanced-security bot found potential problems Jan 3, 2025
View reviewed changes
.github/workflows/yocto-build-deploy.yml Dismissed Show dismissed Hide dismissed
@klutchell klutchell temporarily deployed to balena-staging.com January 13, 2025 19:29 — with GitHub Actions Inactive
@klutchell
Separate build jobs by environments
e41fb95 
This allows more granular control over which environments are used
for which steps, and avoids requiring a single environment to define
how to build, sign, and publish.

This is better for security as it allows each job to only have
the secrets it needs, and not all secrets for all steps.

We retain backwards compatibility by falling back to the original
monolith environment input for now.

Change-type: minor
Signed-off-by: Kyle Harding <[email protected]>
@klutchell klutchell temporarily deployed to balena-staging.com January 13, 2025 20:48 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 13, 2025 20:48 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 13, 2025 20:49 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 13, 2025 21:23 — with GitHub Actions Inactive
@klutchell
Provide new environments as inputs
0b2d291 
Signed-off-by: Kyle Harding <[email protected]>
@klutchell
Remove balena API key from yocto build step
c8de092 
This key was only used to fetch the supervisor image name from the API,
but that endpoint does not need auth as the supervisor images are public.

Change-type: patch
Signed-off-by: Kyle Harding <[email protected]>
@klutchell
Upload all prepared artifacts for deploy jobs
bd2b590 
S3 deploy needs most of the files generated by the
prepare artifacts function, so compress them as much
as possible and remove images that have been zipped.

This creates artifacts up to 2.4gb in size, but most
should be less.

Signed-off-by: Kyle Harding <[email protected]>
@klutchell
Unroll is_private function into github-script
5386e71 
Signed-off-by: Kyle Harding <[email protected]>
@klutchell
Make all jobs depend on the approved-commit job
b435169 
Signed-off-by: Kyle Harding <[email protected]>
@klutchell klutchell marked this pull request as ready for review January 16, 2025 21:09
@klutchell klutchell temporarily deployed to balena-staging.com January 16, 2025 21:09 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 16, 2025 21:09 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com January 16, 2025 21:10 — with GitHub Actions Inactive
@flowzone-app flowzone-app bot enabled auto-merge January 16, 2025 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rcooke-warwick rcooke-warwick Awaiting requested review from rcooke-warwick

@vipulgupta2048 vipulgupta2048 Awaiting requested review from vipulgupta2048

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@klutchell