WIP: fix unbounded data loading

[mvdbeek]

xref #20750

a lot more work here, and for some cases we might benefit from some smarter scanning of json values, like for the zarr case

(Please replace this header with a description of your pull request. Please include BOTH what you did and why you made the changes. The "why" may simply be citing a relevant Galaxy issue.)
(If fixing a bug, please add any relevant error or traceback)
(For UI components, it is recommended to include screenshots or screencasts)

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[ksuderman]

What about using something like json-stream to lazy load the json files? I haven't done a deep dive into the code base, but it might be an option if we only need a few fields from a large json file.