-
Notifications
You must be signed in to change notification settings - Fork 8.5k
[CI] Storybook parallel build #234841
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI] Storybook parallel build #234841
Conversation
6bf4cf3 to
132d91d
Compare
132d91d to
a573fb4
Compare
|
Pinging @elastic/kibana-operations (Team:Operations) |
💚 Build Succeeded
Metrics [docs]
History
cc @Ikuni17 |
|
Starting backport for target branches: 8.19, 9.0, 9.1 |
## Summary - Part of elastic/kibana-operations#346 & elastic/kibana-operations#347 - Closes elastic#176500 - Parallelize Storybook build - Migrate `gsutil` to `gcloud storage cp` ### Testing I tried a few machine sizes to compare the build times: | Machine | Time | Reduction | |--------------|--------------|---------| | `n2-standard-32` | 9m 32s | 82% | | `n2-standard-16` | 15m 43s | 69% | | `n2-standard-8` | 19m 32s | 62% | | `n2-standard-4` | 38m 39s | 24% | | Sync `n2-standard-8` | 50m 40s | - | For now we can stick with `n2-standard-8` since 20 minutes will be well under the FTR times and gives a minor decrease in CI costs since we're using the machine for less time. (cherry picked from commit 29d189f)
## Summary - Part of elastic/kibana-operations#346 & elastic/kibana-operations#347 - Closes elastic#176500 - Parallelize Storybook build - Migrate `gsutil` to `gcloud storage cp` ### Testing I tried a few machine sizes to compare the build times: | Machine | Time | Reduction | |--------------|--------------|---------| | `n2-standard-32` | 9m 32s | 82% | | `n2-standard-16` | 15m 43s | 69% | | `n2-standard-8` | 19m 32s | 62% | | `n2-standard-4` | 38m 39s | 24% | | Sync `n2-standard-8` | 50m 40s | - | For now we can stick with `n2-standard-8` since 20 minutes will be well under the FTR times and gives a minor decrease in CI costs since we're using the machine for less time. (cherry picked from commit 29d189f)
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
## Summary - Part of elastic/kibana-operations#346 & elastic/kibana-operations#347 - Closes elastic#176500 - Parallelize Storybook build - Migrate `gsutil` to `gcloud storage cp` ### Testing I tried a few machine sizes to compare the build times: | Machine | Time | Reduction | |--------------|--------------|---------| | `n2-standard-32` | 9m 32s | 82% | | `n2-standard-16` | 15m 43s | 69% | | `n2-standard-8` | 19m 32s | 62% | | `n2-standard-4` | 38m 39s | 24% | | Sync `n2-standard-8` | 50m 40s | - | For now we can stick with `n2-standard-8` since 20 minutes will be well under the FTR times and gives a minor decrease in CI costs since we're using the machine for less time. (cherry picked from commit 29d189f) # Conflicts: # .buildkite/scripts/steps/storybooks/build_and_upload.ts
# Backport This will backport the following commits from `main` to `9.1`: - [[CI] Storybook parallel build (#234841)](#234841) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Brad White","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-09-17T23:26:47Z","message":"[CI] Storybook parallel build (#234841)\n\n## Summary\n\n- Part of elastic/kibana-operations#346 & elastic/kibana-operations#347\n- Closes #176500\n- Parallelize Storybook build\n- Migrate `gsutil` to `gcloud storage cp`\n\n### Testing\nI tried a few machine sizes to compare the build times:\n\n| Machine | Time | Reduction |\n|--------------|--------------|---------|\n| `n2-standard-32` | 9m 32s | 82% |\n| `n2-standard-16` | 15m 43s | 69% |\n| `n2-standard-8` | 19m 32s | 62% |\n| `n2-standard-4` | 38m 39s | 24% |\n| Sync `n2-standard-8` | 50m 40s | - |\n\nFor now we can stick with `n2-standard-8` since 20 minutes will be well\nunder the FTR times and gives a minor decrease in CI costs since we're\nusing the machine for less time.","sha":"29d189ff7df46d791d503311fff6c06f77c7a271","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","ci:build-storybooks","backport:version","v9.2.0","v8.19.5","v9.0.8","v9.1.5"],"title":"[CI] Storybook parallel build","number":234841,"url":"https://github.com/elastic/kibana/pull/234841","mergeCommit":{"message":"[CI] Storybook parallel build (#234841)\n\n## Summary\n\n- Part of elastic/kibana-operations#346 & elastic/kibana-operations#347\n- Closes #176500\n- Parallelize Storybook build\n- Migrate `gsutil` to `gcloud storage cp`\n\n### Testing\nI tried a few machine sizes to compare the build times:\n\n| Machine | Time | Reduction |\n|--------------|--------------|---------|\n| `n2-standard-32` | 9m 32s | 82% |\n| `n2-standard-16` | 15m 43s | 69% |\n| `n2-standard-8` | 19m 32s | 62% |\n| `n2-standard-4` | 38m 39s | 24% |\n| Sync `n2-standard-8` | 50m 40s | - |\n\nFor now we can stick with `n2-standard-8` since 20 minutes will be well\nunder the FTR times and gives a minor decrease in CI costs since we're\nusing the machine for less time.","sha":"29d189ff7df46d791d503311fff6c06f77c7a271"}},"sourceBranch":"main","suggestedTargetBranches":["8.19","9.0","9.1"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/234841","number":234841,"mergeCommit":{"message":"[CI] Storybook parallel build (#234841)\n\n## Summary\n\n- Part of elastic/kibana-operations#346 & elastic/kibana-operations#347\n- Closes #176500\n- Parallelize Storybook build\n- Migrate `gsutil` to `gcloud storage cp`\n\n### Testing\nI tried a few machine sizes to compare the build times:\n\n| Machine | Time | Reduction |\n|--------------|--------------|---------|\n| `n2-standard-32` | 9m 32s | 82% |\n| `n2-standard-16` | 15m 43s | 69% |\n| `n2-standard-8` | 19m 32s | 62% |\n| `n2-standard-4` | 38m 39s | 24% |\n| Sync `n2-standard-8` | 50m 40s | - |\n\nFor now we can stick with `n2-standard-8` since 20 minutes will be well\nunder the FTR times and gives a minor decrease in CI costs since we're\nusing the machine for less time.","sha":"29d189ff7df46d791d503311fff6c06f77c7a271"}},{"branch":"8.19","label":"v8.19.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.0","label":"v9.0.8","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Brad White <[email protected]>
# Backport This will backport the following commits from `main` to `9.0`: - [[CI] Storybook parallel build (#234841)](#234841) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Brad White","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-09-17T23:26:47Z","message":"[CI] Storybook parallel build (#234841)\n\n## Summary\n\n- Part of elastic/kibana-operations#346 & elastic/kibana-operations#347\n- Closes #176500\n- Parallelize Storybook build\n- Migrate `gsutil` to `gcloud storage cp`\n\n### Testing\nI tried a few machine sizes to compare the build times:\n\n| Machine | Time | Reduction |\n|--------------|--------------|---------|\n| `n2-standard-32` | 9m 32s | 82% |\n| `n2-standard-16` | 15m 43s | 69% |\n| `n2-standard-8` | 19m 32s | 62% |\n| `n2-standard-4` | 38m 39s | 24% |\n| Sync `n2-standard-8` | 50m 40s | - |\n\nFor now we can stick with `n2-standard-8` since 20 minutes will be well\nunder the FTR times and gives a minor decrease in CI costs since we're\nusing the machine for less time.","sha":"29d189ff7df46d791d503311fff6c06f77c7a271","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","ci:build-storybooks","backport:version","v9.2.0","v8.19.5","v9.0.8","v9.1.5"],"title":"[CI] Storybook parallel build","number":234841,"url":"https://github.com/elastic/kibana/pull/234841","mergeCommit":{"message":"[CI] Storybook parallel build (#234841)\n\n## Summary\n\n- Part of elastic/kibana-operations#346 & elastic/kibana-operations#347\n- Closes #176500\n- Parallelize Storybook build\n- Migrate `gsutil` to `gcloud storage cp`\n\n### Testing\nI tried a few machine sizes to compare the build times:\n\n| Machine | Time | Reduction |\n|--------------|--------------|---------|\n| `n2-standard-32` | 9m 32s | 82% |\n| `n2-standard-16` | 15m 43s | 69% |\n| `n2-standard-8` | 19m 32s | 62% |\n| `n2-standard-4` | 38m 39s | 24% |\n| Sync `n2-standard-8` | 50m 40s | - |\n\nFor now we can stick with `n2-standard-8` since 20 minutes will be well\nunder the FTR times and gives a minor decrease in CI costs since we're\nusing the machine for less time.","sha":"29d189ff7df46d791d503311fff6c06f77c7a271"}},"sourceBranch":"main","suggestedTargetBranches":["8.19","9.0","9.1"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/234841","number":234841,"mergeCommit":{"message":"[CI] Storybook parallel build (#234841)\n\n## Summary\n\n- Part of elastic/kibana-operations#346 & elastic/kibana-operations#347\n- Closes #176500\n- Parallelize Storybook build\n- Migrate `gsutil` to `gcloud storage cp`\n\n### Testing\nI tried a few machine sizes to compare the build times:\n\n| Machine | Time | Reduction |\n|--------------|--------------|---------|\n| `n2-standard-32` | 9m 32s | 82% |\n| `n2-standard-16` | 15m 43s | 69% |\n| `n2-standard-8` | 19m 32s | 62% |\n| `n2-standard-4` | 38m 39s | 24% |\n| Sync `n2-standard-8` | 50m 40s | - |\n\nFor now we can stick with `n2-standard-8` since 20 minutes will be well\nunder the FTR times and gives a minor decrease in CI costs since we're\nusing the machine for less time.","sha":"29d189ff7df46d791d503311fff6c06f77c7a271"}},{"branch":"8.19","label":"v8.19.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.0","label":"v9.0.8","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Brad White <[email protected]>
# Backport This will backport the following commits from `main` to `8.19`: - [[CI] Storybook parallel build (#234841)](#234841) - [Remove build hash from Storybook upload (#206687)](#206687) <!--- Backport version: 10.0.2 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Brad White","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-09-17T23:26:47Z","message":"[CI] Storybook parallel build (#234841)\n\n## Summary\n\n- Part of elastic/kibana-operations#346 & elastic/kibana-operations#347\n- Closes #176500\n- Parallelize Storybook build\n- Migrate `gsutil` to `gcloud storage cp`\n\n### Testing\nI tried a few machine sizes to compare the build times:\n\n| Machine | Time | Reduction |\n|--------------|--------------|---------|\n| `n2-standard-32` | 9m 32s | 82% |\n| `n2-standard-16` | 15m 43s | 69% |\n| `n2-standard-8` | 19m 32s | 62% |\n| `n2-standard-4` | 38m 39s | 24% |\n| Sync `n2-standard-8` | 50m 40s | - |\n\nFor now we can stick with `n2-standard-8` since 20 minutes will be well\nunder the FTR times and gives a minor decrease in CI costs since we're\nusing the machine for less time.","sha":"29d189ff7df46d791d503311fff6c06f77c7a271","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","ci:build-storybooks","backport:version","v9.2.0","v8.19.5","v9.0.8","v9.1.5"],"title":"[CI] Storybook parallel build","number":234841,"url":"https://github.com/elastic/kibana/pull/234841","mergeCommit":{"message":"[CI] Storybook parallel build (#234841)\n\n## Summary\n\n- Part of elastic/kibana-operations#346 & elastic/kibana-operations#347\n- Closes #176500\n- Parallelize Storybook build\n- Migrate `gsutil` to `gcloud storage cp`\n\n### Testing\nI tried a few machine sizes to compare the build times:\n\n| Machine | Time | Reduction |\n|--------------|--------------|---------|\n| `n2-standard-32` | 9m 32s | 82% |\n| `n2-standard-16` | 15m 43s | 69% |\n| `n2-standard-8` | 19m 32s | 62% |\n| `n2-standard-4` | 38m 39s | 24% |\n| Sync `n2-standard-8` | 50m 40s | - |\n\nFor now we can stick with `n2-standard-8` since 20 minutes will be well\nunder the FTR times and gives a minor decrease in CI costs since we're\nusing the machine for less time.","sha":"29d189ff7df46d791d503311fff6c06f77c7a271"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/234841","number":234841,"mergeCommit":{"message":"[CI] Storybook parallel build (#234841)\n\n## Summary\n\n- Part of elastic/kibana-operations#346 & elastic/kibana-operations#347\n- Closes #176500\n- Parallelize Storybook build\n- Migrate `gsutil` to `gcloud storage cp`\n\n### Testing\nI tried a few machine sizes to compare the build times:\n\n| Machine | Time | Reduction |\n|--------------|--------------|---------|\n| `n2-standard-32` | 9m 32s | 82% |\n| `n2-standard-16` | 15m 43s | 69% |\n| `n2-standard-8` | 19m 32s | 62% |\n| `n2-standard-4` | 38m 39s | 24% |\n| Sync `n2-standard-8` | 50m 40s | - |\n\nFor now we can stick with `n2-standard-8` since 20 minutes will be well\nunder the FTR times and gives a minor decrease in CI costs since we're\nusing the machine for less time.","sha":"29d189ff7df46d791d503311fff6c06f77c7a271"}},{"branch":"8.19","label":"v8.19.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.0","label":"v9.0.8","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/235436","number":235436,"state":"OPEN"},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/235437","number":235437,"state":"OPEN"}]}] BACKPORT-->
## Summary - Part of elastic/kibana-operations#346 & elastic/kibana-operations#347 - Closes elastic#176500 - Parallelize Storybook build - Migrate `gsutil` to `gcloud storage cp` ### Testing I tried a few machine sizes to compare the build times: | Machine | Time | Reduction | |--------------|--------------|---------| | `n2-standard-32` | 9m 32s | 82% | | `n2-standard-16` | 15m 43s | 69% | | `n2-standard-8` | 19m 32s | 62% | | `n2-standard-4` | 38m 39s | 24% | | Sync `n2-standard-8` | 50m 40s | - | For now we can stick with `n2-standard-8` since 20 minutes will be well under the FTR times and gives a minor decrease in CI costs since we're using the machine for less time.
## Summary - Part of elastic/kibana-operations#346 & elastic/kibana-operations#347 - Closes #176500 - Parallelize Storybook build - Migrate `gsutil` to `gcloud storage cp` ### Testing I tried a few machine sizes to compare the build times: | Machine | Time | Reduction | |--------------|--------------|---------| | `n2-standard-32` | 9m 32s | 82% | | `n2-standard-16` | 15m 43s | 69% | | `n2-standard-8` | 19m 32s | 62% | | `n2-standard-4` | 38m 39s | 24% | | Sync `n2-standard-8` | 50m 40s | - | For now we can stick with `n2-standard-8` since 20 minutes will be well under the FTR times and gives a minor decrease in CI costs since we're using the machine for less time.
## Summary - Part of elastic/kibana-operations#346 & elastic/kibana-operations#347 - Closes elastic#176500 - Parallelize Storybook build - Migrate `gsutil` to `gcloud storage cp` ### Testing I tried a few machine sizes to compare the build times: | Machine | Time | Reduction | |--------------|--------------|---------| | `n2-standard-32` | 9m 32s | 82% | | `n2-standard-16` | 15m 43s | 69% | | `n2-standard-8` | 19m 32s | 62% | | `n2-standard-4` | 38m 39s | 24% | | Sync `n2-standard-8` | 50m 40s | - | For now we can stick with `n2-standard-8` since 20 minutes will be well under the FTR times and gives a minor decrease in CI costs since we're using the machine for less time.
Summary
gsutiltogcloud storage cpTesting
I tried a few machine sizes to compare the build times:
n2-standard-32n2-standard-16n2-standard-8n2-standard-4n2-standard-8For now we can stick with
n2-standard-8since 20 minutes will be well under the FTR times and gives a minor decrease in CI costs since we're using the machine for less time.