Skip to content

Add --diff argument to benchmark-web-vitals and benchmark-server-timing #187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 25, 2025

Conversation

westonruter
Copy link
Collaborator

This is a follow-up to #186.

The most common use case for benchmarking two URLs is to see the performance impact before/after. To do this, I would often copy the CSV output into a Google Sheet and then add columns to compute the difference in terms of milliseconds and percentage. Since this is such a common scenario (at least for me), this PR adds support for passing --diff when calling benchmark-web-vitals or benchmark-server-timing, if the --file arg is referring to a file that contains two URLs.

Example output:

npm run research -- benchmark-web-vitals --file="urls.txt" --number=100 --output="md" --network-conditions="broadband" --diff
URL http://localhost:10008/bison/?disable_script_fetchpriority_low=1 http://localhost:10008/bison/ Diff (ms) Diff (%)
Success Rate 100% 100% -- --
FCP (median) 142.55 141.7 -0.85 -0.6%
LCP (median) 409.35 382.35 -27.00 -6.6%
TTFB (median) 34.65 35.3 +0.65 +1.9%
LCP-TTFB (median) 374.15 347 -27.15 -7.3%

@westonruter westonruter added the enhancement New feature or request label May 21, 2025
@westonruter westonruter requested a review from felixarntz May 21, 2025 18:45
… into add/benchmark-diff

* 'main' of https://github.com/GoogleChromeLabs/wpp-research:
  Throw error instead of logging
  Remove needless variable defined outside try/catch
  Catch puppeteer errors to count toward non-success rate
@westonruter westonruter requested a review from swissspidy May 23, 2025 17:41
… into add/benchmark-diff

* 'main' of https://github.com/GoogleChromeLabs/wpp-research:
  Fix lint errors
  Allow passing multiple URLs by repeating the --url arg when benchmarking
  Update commander to v14
  Show error when all benchmark requests for a URL failed
  Ensure browser is closed when network priming request fails
  Prevent terminal from linkifying ... as part of the logged URL
Copy link
Collaborator

@swissspidy swissspidy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it! 👍

@westonruter westonruter merged commit be3554e into main May 25, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants