Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Goals first: Use actual experiment checks within useGoalsFirstExperiment #97699

Open
wants to merge 2 commits into
base: trunk
Choose a base branch
from

Conversation

zaguiini
Copy link
Contributor

@zaguiini zaguiini commented Dec 20, 2024

Closes #96179.

Proposed Changes

Remove the flag onboarding/goals-first and use the actual experiment name and variant names.

@zaguiini zaguiini requested a review from a team December 20, 2024 13:31
@zaguiini zaguiini self-assigned this Dec 20, 2024
@matticbot matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Dec 20, 2024
@matticbot
Copy link
Contributor

This PR modifies the release build for the following Calypso Apps:

For info about this notification, see here: PCYsg-OT6-p2

  • notifications
  • wpcom-block-editor

To test WordPress.com changes, run install-plugin.sh $pluginSlug goals-first/use-experiment on your sandbox.

@zaguiini zaguiini force-pushed the goals-first/use-experiment branch from fb6fc54 to a19a04c Compare December 20, 2024 13:40
@matticbot
Copy link
Contributor

matticbot commented Dec 20, 2024

Here is how your PR affects size of JS and CSS bundles shipped to the user's browser:

App Entrypoints (~1 bytes added 📈 [gzipped])

name           parsed_size           gzip_size
entry-stepper        +42 B  (+0.0%)       +1 B  (+0.0%)

Common code that is always downloaded and parsed every time the app is loaded, no matter which route is used.

Sections (~4736 bytes added 📈 [gzipped])

name                                parsed_size           gzip_size
onboarding-flow                         +6799 B  (+7.1%)    +2651 B  (+20.8%)
import-hosted-site-flow                 +6679 B  (+0.6%)    +2278 B   (+0.7%)
copy-site-flow                          +6679 B  (+0.7%)    +2245 B   (+0.8%)
link-in-bio-tld-flow                     -323 B  (-0.0%)      +14 B   (+0.0%)
signup                                   -264 B  (-0.1%)      -10 B   (-0.0%)
update-design-flow                       -237 B  (-0.0%)       -4 B   (-0.0%)
themes                                    -59 B  (-0.0%)      +22 B   (+0.0%)
theme                                     -59 B  (-0.0%)      +25 B   (+0.0%)
tailored-ecommerce-flow                   +54 B  (+0.1%)      +14 B   (+0.2%)
with-theme-assembler-flow                 +27 B  (+0.0%)       +5 B   (+0.0%)
update-options-flow                       +27 B  (+0.0%)       +5 B   (+0.1%)
trial-wooexpress-flow                     +27 B  (+0.0%)       +5 B   (+0.1%)
start-writing-flow                        +27 B  (+0.1%)      +12 B   (+0.1%)
site-setup-wg                             +27 B  (+0.0%)       +5 B   (+0.0%)
site-setup-flow                           +27 B  (+0.0%)       +5 B   (+0.0%)
site-migration-flow                       +27 B  (+0.0%)       +5 B   (+0.0%)
reblogging-flow                           +27 B  (+0.4%)      +10 B   (+0.5%)
readymade-template-flow                   +27 B  (+0.0%)       +5 B   (+0.0%)
newsletter-flow                           +27 B  (+0.1%)       +9 B   (+0.1%)
new-hosted-site-flow-user-included        +27 B  (+0.4%)       +7 B   (+0.3%)
new-hosted-site-flow                      +27 B  (+0.2%)       +8 B   (+0.3%)
migration-signup                          +27 B  (+0.0%)       +5 B   (+0.0%)
migration-flow                            +27 B  (+0.0%)       +5 B   (+0.0%)
import-flow                               +27 B  (+0.0%)       +5 B   (+0.0%)
hundred-year-plan                         +27 B  (+0.1%)      +12 B   (+0.1%)
hosted-site-migration-flow                +27 B  (+0.0%)       +5 B   (+0.0%)
free-post-setup-flow                      +27 B  (+0.0%)       +5 B   (+0.1%)
entrepreneur-flow                         +27 B  (+0.0%)       +5 B   (+0.0%)
design-first-flow                         +27 B  (+0.1%)      +12 B   (+0.1%)
connect-domain                            +27 B  (+0.1%)       +7 B   (+0.1%)
assembler-first-flow                      +27 B  (+0.0%)       +5 B   (+0.0%)
ai-assembler-flow                         +27 B  (+0.0%)       +5 B   (+0.0%)

Sections contain code specific for a given set of routes. Is downloaded and parsed only when a particular route is navigated to.

Async-loaded Components (~2634 bytes removed 📉 [gzipped])

name                                             parsed_size           gzip_size
async-load-signup-steps-plans-theme-preselected      -6831 B  (-1.5%)    -2656 B  (-1.8%)
async-load-signup-steps-plans                        -6831 B  (-1.5%)    -2656 B  (-1.8%)
async-load-signup-steps-page-picker                  -6831 B  (-1.6%)    -2656 B  (-2.0%)
async-load-signup-steps-theme-selection                -59 B  (-0.0%)      +22 B  (+0.0%)
async-load-design-blocks                               -59 B  (-0.0%)      +22 B  (+0.0%)

React components that are loaded lazily, when a certain part of UI is displayed for the first time.

Legend

What is parsed and gzip size?

Parsed Size: Uncompressed size of the JS and CSS files. This much code needs to be parsed and stored in memory.
Gzip Size: Compressed size of the JS and CSS files. This much data needs to be downloaded over network.

Generated by performance advisor bot at iscalypsofastyet.com.

Copy link
Contributor

@paulopmt1 paulopmt1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working as expected!

When connecting it to the running experiment 22021-explat-experiment (for testing purposes since we'll create a new experiment), it will redirect control users to the new flow and holdout to the old one.

I still recommend a new check once the oficial experiment is in place.

@zaguiini zaguiini force-pushed the goals-first/use-experiment branch from a19a04c to 39d0758 Compare December 23, 2024 20:21
@zaguiini zaguiini marked this pull request as ready for review December 23, 2024 20:22
@zaguiini zaguiini requested a review from a team as a code owner December 23, 2024 20:22
Copy link
Contributor

@candy02058912 candy02058912 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes LGTM 👍

@alshakero
Copy link
Member

I'd test this thoroughly because I have seen issues happen with Stepper's useSteps hook when it returns dynamic results. Consider that Stepper redirects to the first step of the flow when you visit /flowName. So if the experiment returns control, the first step will be different than when it does treatment. And the experiment may load well after Stepper had already redirected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DO NOT MERGE [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Set up experiment holdout for new goals step position
5 participants