Verdaccio is slow listing packages

[edclement]

After updating Verdaccio to use this verdaccio-google-cloud-storage plugin, the time it takes to load the package list (and thus the web UI) has increased into the 3-6 second range. After investigation, this is due to:

1. Everytime the package list is read the full list of package names is read from the GCP Datastore.
2. Once the package list is read, the latest package.json file is pulled from GCP bucket. This happens sequentially instead of in parallel.

This issue cannot be fixed in the storage plugin. It needs to re fixed in the core Verdaccio code. Overall, the way Verdaccio handles loading of data could use a lot of optimization. A key function that could use refactoring is the getLocalDatabase member in the src/lib/storage.ts. This is the function that indirectly makes the calls mentioned in bullet points #1 and #2 above and carries out the reading of the package.json files in a sequential manner with a recursive callback function. You can see the code for that here. Specifically line 433 where the function calls itself only after the previous package.json file has been retrieved.