Multi sources with transforms

[will-moore]

This is an investigation into supporting overlays of multiple images in vizarr, with the support of coordinateTransformations to allow relative positioning and scaling.

Test at https://deploy-preview-259--vizarr.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0101A/13457537.zarr&source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0101A/13457227.zarr

Multiple source query parameters are supported for loading multiple images: `?source=url1&source=url2

E.g. using the sample images from idr0101: https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0101A/13457537.zarr and https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0101A/13457227.zarr which have scale and transform metadata, we can see them overlaid as expected.

If there is any matrix specified in the URL e.g. &model_matrix=[0.5403023058681398,0.8414709848078965,0,0,-0.8414709848078965,0.5403023058681398,0,0,0,0,1,0,0,0,0,1], the coordinateTransformations are applied on top of this.

A couple questions/issues:

• Since the addImage() calls happend asynchronously, we can't reliably control which image appears "on top" of the other. Refreshing the page can lead to the images displayed in different orders.
• The images use the same XY coordinates as each other, but Z and T are independent for each image (each has its own Z/T sliders). The Z translation and scaling are ignored when parsing coordinateTransformations.
  In the example you can see that the smaller image has fewer Z sections than the larger one, but it's not possible to "align" them in any way.

[manzt]

Hey @will-moore, thanks for the PR! I added support for building up the coordinateTransformations in #242. https://github.com/hms-dbmi/vizarr/blob/manzt%2Flabels/src/utils.ts#L529-L579

Would you mind if I made a PR cherry picking these changes in a separate PR and maybe we focus this PR on multiple sources? The utility is from Vitessce and been used for some time. Also #242 has some changes to ensure that the the the initial zoom state is aware of the modelMatrx, https://github.com/hms-dbmi/vizarr/blob/manzt%2Flabels/src/utils.ts#L224-L256

[will-moore]

@manzt Great, thanks. Go ahead 👍

[manzt]

Ok, I merged #261 and rebased #242 on those changes. That really cleaned up the diff for that PR.

Perhaps here we could revert the changes other than in main.ts? I can start to think about how to best support adding multiples images at a time. One idea might just to have addImage somehow return a promise we can await, or have an addImages API to take a list.

  for (const source of sources) {
    await viewer.addImage({ source });
  }

In general, it would be nice to think a bit about the URL API we really want for the website.

[will-moore]

OK, I pushed just my first commit on top of main.

Definitely thinking about url structure. If you wanted to add different config for each source then how would that look?

await viewer.addImage({ source }); would be 👍

[will-moore]

@manzt I don't know if you've had time to think about URL API, but I just pushed a commit that at least allows you to specify different config values for each source image, without breaking the existing URL structure.

E.g. if you have ?source=image1.zarr&opacity=1.0&source=image2.zarr&opacity=0.25 then image1.zarr will get opacity=1.0 and image2.zarr will get opacity=0.25.

E.g. https://deploy-preview-259--vizarr.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0101A/13457227.zarr&source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0062A/6001240.zarr&opacity=1.0&opacity=0.25 (screenshot below)

This relies on searchParams.getAll("value") etc returning values in the order they appear in the URL, which seems to work across all the browsers I've tested (Chrome, Safari, Firefox) but I haven't seen that specified anywhere yet... https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams/getAll

Do you have some docs on what are all the config values supported in the URL query params?
Ah, I see some examples at the end of https://github.com/hms-dbmi/vizarr/blob/main/python/notebooks/getting_started.ipynb