ci: rc, release, tagged rc, hotfix workflows #12918

awsluja · 2023-07-07T20:43:32Z

Description of changes

Issue #, if available

Description of how you validated changes

Checklist

PR description included
yarn test passes
Tests are changed or added
Relevant documentation is changed or added (and PR referenced)
New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies
Pull request labels are added

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sobolk

it seems that publish-codebuild.sh does not reflect changes made on CCI to split the deployment step into pieces.

This PR should either incorporate these changes OR release should be descoped from here and added later with the split.

sobolk

Did get a chance to test it? We should at least get successful tagged release using this workflow.

Major concerns:

We should decouple CB release workflows from CCI sunset. These should be separate changes. Ability to use both systems should exists until we gain confidence in CB. In event of emergency we should be able to buy CCI credits and revert back to old systems should CB prevent us from shipping. Smooth migration must include period where both old and new system is accessible.
The necessity of codebuild-checkout in release steps 2,3,4. Revised release process assumes that outcome of build step is cached including git metadata. Can we get this fixed so that checkout is necessary only in entry steps (build, build-tests-standalone? )

sobolk · 2023-07-28T21:32:21Z

.circleci/config.base.yml

+# version: 2.1
+# orbs:


disabling CCI should be separate PR.

Yep, we shouldn't turn off CCI until we've done at least a couple releases using CodeBuild

sobolk · 2023-07-28T21:32:31Z

.circleci/config.yml

+# version: 2.1

-# this allows you to use CircleCI's dynamic configuration feature
-setup: true
+# # this allows you to use CircleCI's dynamic configuration feature


sobolk · 2023-07-28T21:33:51Z

.circleci/local_publish_helpers_codebuild.sh

+git checkout $BRANCH_NAME
+


We shouldn't checkout a branch every time we load this script.
Is this intentional?

sobolk · 2023-07-28T21:35:55Z

.circleci/local_publish_helpers_codebuild.sh

-        aws --profile=s3-uploader s3 cp amplify-pkg-linux-arm64.tgz s3://aws-amplify-cli-do-not-delete/$(echo $version)/amplify-pkg-linux-arm64-$(echo $hash).tgz
-        aws --profile=s3-uploader s3 cp amplify-pkg-linux-x64.tgz s3://aws-amplify-cli-do-not-delete/$(echo $version)/amplify-pkg-linux-x64-$(echo $hash).tgz
+    if [[ "$BRANCH_NAME" == "release" ]] || [[ "$BRANCH_NAME" =~ ^run-e2e-with-rc\/.* ]] || [[ "$BRANCH_NAME" =~ ^release_rc\/.* ]] || [[ "$BRANCH_NAME" =~ ^tagged-release ]]; then
+        aws s3 cp amplify-pkg-win-x64.tgz s3://$PKG_CLI_BUCKET_NAME/$(echo $version)/amplify-pkg-win-x64-$(echo $hash).tgz


Can we make a copy of this file instead? and drop non-CB version together with CCI later?

Please do this for all other .sh files where we made such assumption.

sobolk · 2023-07-28T21:42:00Z

.circleci/publish-step-2-verdaccio.sh

+source ./.circleci/codebuild-checkout.sh
+


This is unexpected.
In CCI .circleci/publish-step-1-set-versions.sh adds a commit to local repo that is NOT pushed to git.
That commit is retained in other jobs because we cache result of build step, including .git directory which preserves that information. We should not need to checkout everywhere if we we cache workspace properly.

This is concern for other files that attempt to checkout after build step.

sobolk · 2023-07-28T21:43:48Z

codebuild_specs/release_workflows/release_workflow.yml

+      buildspec: codebuild_specs/release_workflows/github_release.yml
+      env:
+        compute-type: BUILD_GENERAL1_LARGE
+      debug-session: true


do we want these debug sessions ?

this is just part of it; the other part is to manually trigger the job with debug sessions enabled & ssm enabled

codecov-commenter · 2023-08-03T03:04:15Z

Codecov Report

Merging #12918 (c816f02) into dev (ec9a2ba) will increase coverage by 48.56%.
Report is 288 commits behind head on dev.
The diff coverage is 58.82%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@             Coverage Diff             @@
##              dev   #12918       +/-   ##
===========================================
+ Coverage    0.00%   48.56%   +48.56%     
===========================================
  Files        1296      848      -448     
  Lines      149743    38131   -111612     
  Branches     1296     7798     +6502     
===========================================
+ Hits            0    18519    +18519     
+ Misses     148447    18022   -130425     
- Partials     1296     1590      +294

Files Changed	Coverage Δ
...rmation/auth-stack-builder/auth-stack-transform.ts	`91.30% <0.00%> (+91.30%)`	⬆️
...tack-builder/auth-user-pool-group-stack-builder.ts	`51.72% <ø> (+51.72%)`	⬆️
...h-stack-builder/user-pool-group-stack-transform.ts	`94.59% <0.00%> (+94.59%)`	⬆️
.../src/provider-utils/awscloudformation/constants.ts	`100.00% <ø> (+100.00%)`	⬆️
...e-walkthrough-types/awsCognito-user-input-types.ts	`100.00% <ø> (+100.00%)`	⬆️
...vice-walkthrough-types/cognito-user-input-types.ts	`100.00% <ø> (+100.00%)`	⬆️
...y-category-function/src/commands/function/build.ts	`0.00% <0.00%> (ø)`
...ify-category-function/src/events/prePushHandler.ts	`33.33% <0.00%> (+33.33%)`	⬆️
...ider-utils/awscloudformation/utils/layerHelpers.ts	`21.80% <0.00%> (+21.80%)`	⬆️
...er-utils/awscloudformation/utils/storeResources.ts	`30.38% <ø> (+30.38%)`	⬆️
... and 52 more

... and 1249 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

sobolk

LGTM.

sobolk · 2023-08-03T14:33:05Z

codebuild_specs/amplify_general_config_tests.yml

+artifacts:
+  files:
+    - 'shared-scripts.sh'


what is this for ?

codebuild requires a job to output an artifact, otherwise other jobs cannot depend on it downstream

sobolk · 2023-08-03T14:34:55Z

codebuild_specs/publish_to_npm.yml

+version: 0.2
+env:
+  shell: bash
+  git-credential-helper: yes


how doesgit-credential-helper work ?

only the jobs that require .git credentials should use git-credential-helper; this injects the GH credentials via the git-credential-helper https://git-scm.com/docs/gitcredentials so that things like push work. Can only be used on jobs that are manually triggered via console/api.

.circleci/cb-publish-step-3-npm.sh

jhockett · 2023-08-03T16:32:25Z

shared-scripts.sh

    # if we run into problems in the future, we should revisit this
-    # git reset --soft HEAD~1
+    git reset --soft HEAD~1


What's the goal of doing a soft reset?

if i remember correctly; its to remove the commit that is created during publish to verdaccio step so that it doesn't affect the unified changelog step

this shouldn't be necessary. does CCI do this ?

or maybe it does make some sense, should we unreset after step which is affected by this ?

I see, this unified changelog script is designed to work with staged files:

// This script is intended to be run after running `yarn publish-to-verdaccio` when all CHANGELOG file changes are staged but not committed // It will scan all staged CHANGELOG.md files and merge them into a single UNIFIED_CHANGELOG file

ci: rc, release, tagged rc, hotfix workflows

e269f60

awsluja requested review from a team as code owners July 7, 2023 20:43

sobolk requested changes Jul 7, 2023

View reviewed changes

awsluja mentioned this pull request Jul 7, 2023

DRAFT: release pipelines configurations #12570

Closed

6 tasks

awsluja added 9 commits July 7, 2023 19:24

Merge branch 'dev' into codebuild-release-workflows

772c656

ci: cleanup

2b456f5

ci: update prepush hooks

bb30638

ci: adjustments for yarn2

4d976ec

Merge branch 'dev' into codebuild-release-workflows

62851e4

ci: split deployment step codebuild

719fdfc

ci: remove references to old helpers file

6af6e8c

ci: remove old publish helpers file

3820f6a

ci: add artifact to allow all jobs to be dependedon

cd625f7

sobolk reviewed Jul 28, 2023

View reviewed changes

awsluja added 12 commits July 28, 2023 19:06

ci: add deployment verification steps

dc947a2

ci: enable tagged release

4c6daf4

ci: fix typo in path to file

09247c5

Merge branch 'dev' into codebuild-release-workflows

ebb688e

Merge branch 'dev' into codebuild-release-workflows

8416b4c

ci: aea flag

4fe8b7b

ci: try source

86c67ce

ci: fix permissions on file

d0d4dc3

ci: remove checkout in publish helpers

3594891

ci: cleanup extra step

dbcaa1f

ci: maintain old cci files

398f3ac

Merge branch 'dev' into codebuild-release-workflows

c816f02

sobolk approved these changes Aug 3, 2023

View reviewed changes

jhockett reviewed Aug 3, 2023

View reviewed changes

jhockett approved these changes Aug 3, 2023

View reviewed changes

awsluja merged commit 71257ec into aws-amplify:dev Aug 3, 2023
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: rc, release, tagged rc, hotfix workflows #12918

ci: rc, release, tagged rc, hotfix workflows #12918

awsluja commented Jul 7, 2023

sobolk left a comment

sobolk left a comment

sobolk Jul 28, 2023

edwardfoyle Jul 28, 2023

sobolk Jul 28, 2023

sobolk Jul 28, 2023

sobolk Jul 28, 2023

awsluja Aug 3, 2023

sobolk Jul 28, 2023

sobolk Jul 28, 2023

awsluja Aug 3, 2023

codecov-commenter commented Aug 3, 2023

sobolk left a comment

sobolk Aug 3, 2023

awsluja Aug 3, 2023

sobolk Aug 3, 2023

sobolk Aug 3, 2023

awsluja Aug 3, 2023

jhockett Aug 3, 2023

awsluja Aug 3, 2023

sobolk Aug 3, 2023

sobolk Aug 3, 2023

jhockett Aug 3, 2023

		source ./.circleci/codebuild-checkout.sh

ci: rc, release, tagged rc, hotfix workflows #12918

ci: rc, release, tagged rc, hotfix workflows #12918

Conversation

awsluja commented Jul 7, 2023

Description of changes

Issue #, if available

Description of how you validated changes

Checklist

sobolk left a comment

Choose a reason for hiding this comment

sobolk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 3, 2023

Codecov Report

sobolk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment