Delete all non-en-US builds?

[peterbe]

When we built Buildhub2 we not only changed the stack (still Python and still React SearchKit) we also entirely changed the architecture such that we only store and index created *buildhub.json files instead of trying to figure out what the build should be based on the binaries + metadata. Because of that only get en-US builds.

https://bugzilla.mozilla.org/show_bug.cgi?id=1459302 is about adding the en-US builds additionally but that bug is stalled (without momentum or owners) and maybe that's OK. The people who have come forth and shown interest in this project are all OK with just en-US. They just need the versions and dates and the buildIDs.

The reason there are non-en-US builds at all in https://buildhub.moz.tools is because of the legacy migration from https://mozilla-services.github.io/buildhub/ (aka. Buildhub1). If we're not bothering with them going forward, what's the point of bothering with it in the past.

We're not running low on disk-space but the feeling of dropping nearly 100x of the data is attractive.

[willkg]

Socorro has its own archive.mozilla.org scraper and captures data for the BetaVersionRule.

The only place where Socorro uses Buildhub is links in the report view to Buildhub so that engineers can get additional information about that build and binaries related to the crash in question. I have no idea if engineers ever need to get non-en-US binaries. That's the only concern I can think of for Socorro.

[wlach]

Yeah, Mission Control only uses the en-US data, or really the subset of that which isn't locale-specific. This is ok by me.

[peterbe]

@willkg Socorro links to the buildID and that's the same for en-US as it is for sv-SE or fr. The only thing that is different between en-US and, say, de is that the download URL (and download size) is different. E.g. https://archive.mozilla.org/pub/firefox/releases/66.0b4/win64/en-US/Firefox%20Setup%2066.0b4.exe vs https://archive.mozilla.org/pub/firefox/releases/66.0b4/win64/de/Firefox%20Setup%2066.0b4.exe

[mars-f]

I only need en-US for build systems trend analysis. This is OK by me.

[fbertsch]

This is completely fine for my use-case - we only need distinct revisions for every release date. I was going to filter on en-US anyways.

[peterbe]

@fbertsch Without fully comprehending your usecase, if you can find a way to use Elasticsearch aggregates to get a distinct list of revisions you'd never need to do any filtering by locale anyway.

[flodolo]

No issues on the l10n side about removing non-en-US build. We weren't aware that this data existed, and there are no real use cases driving the work to add support for l10n builds, as far as I can think of it.