Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regenerate data more often #87

Merged
merged 1 commit into from
May 4, 2022

Conversation

gsnedders
Copy link
Member

Now we're regenerating everything from scratch on each run, including BSF, there's no reason to not regenerate everything more often. The only downside here is that we will be changing the results for the yesterday/today on multiple occasions, rather than the figures being mostly constant once they're added.

See also #80/#81 where we previously landed/reverted this. The only difference here is we run git-write.js at the start of build.sh, to make sure the local copy of the results has been updated as recently as possible. (Of course, because the various scripts call the wpt.fyi API themselves there is still a risk of a new aligned run appearing between running git-write.js and the actual scoring scripts, but that's relatively unlikely.) Ideally we'd update wpt-results-cache on demand from within each script as needed, but that's a much larger change, and also requires dealing with secrets to push to the other repo. This is almost certainly good enough for the vast majority of scheduled jobs, and even if the odd one fails we'll still have more up-to-date results than we had previously.

Now we're regenerating everything from scratch on each run, including BSF,
there's no reason to not regenerate everything more often. The only downside
here is that we will be changing the results for the yesterday/today on multiple
occasions, rather than the figures being mostly constant once they're added.
@gsnedders
Copy link
Member Author

Oh, no, we need to change the caching of aligned runs in at least browser-specific-failures.js too.

@foolip
Copy link
Member

foolip commented May 4, 2022

Ideally we'd update wpt-results-cache on demand from within each script as needed, but that's a much larger change, and also requires dealing with secrets to push to the other repo. This is almost certainly good enough for the vast majority of scheduled jobs, and even if the odd one fails we'll still have more up-to-date results than we had previously.

Just loading the results into memory from a wptreport.json and not bothering pushing that to wpt-results-cache would avoid the secrets problem, but I agree this is good enough as-is.

@foolip foolip merged commit 84143f5 into web-platform-tests:main May 4, 2022
gsnedders added a commit to gsnedders/results-analysis that referenced this pull request Aug 9, 2022
This regressed with web-platform-tests#87, but it doesn't occur in CI as we never have a
populated cache when this is run.
gsnedders added a commit to gsnedders/results-analysis that referenced this pull request Aug 9, 2022
This regressed with web-platform-tests#87, but it doesn't occur in CI as we never have a
populated cache when this is run.
foolip pushed a commit that referenced this pull request Aug 18, 2022
Avoid caching the last few days in case new runs appear.

This regressed with #87, but it doesn't occur in CI as we never have a
populated cache when this is run.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants