discoverable axes index lookup #14
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| # Axes lookup datastructure | ||
|
|
||
| ``` | ||
| \ | ||
| ├── ".axes" | ||
| │ ├── "some2DVector" | ||
| │ │ ├── attributes.json {"dataType":"int32","compression":{"type":"raw"},"blockSize":[2,2],"dimensions":[2,2]} | ||
| │ ├── "a3DVector" | ||
| │ │ ├── attributes.json {"dataType":"int32","compression":{"type":"raw"},"blockSize":[2,2,2],"dimensions":[2,2,2]} | ||
| │ ┊ | ||
| │ | ||
| ├── ... | ||
| ┊ | ||
| ``` | ||
|
|
||
| Each "vector" dataset is a 2^n size single block dataset that enumerates the axes indices. The 2D axes lookup for ImgLib2 vectors (F-order) looks like this: | ||
|
|
||
| ``` | ||
| -1 0 -> -1 0 1 -1 | ||
| 1 -1 | ||
| ``` | ||
|
There was a problem hiding this comment. Is there a more detailed specification of what and mean? I have a hard time understanding how they are generated. There was a problem hiding this comment. Looking at the source code helps a lot (thanks @igorpisarev for pointing out the correct names), so maybe There was a problem hiding this comment. Agree but currenty have no better idea. Will work on it. |
||
|
|
||
| The 3D axes lookup for numpy vectors (C-order) looks like this: | ||
|
|
||
| ``` | ||
| -1 2 -> -1 2 1 -1 0 -1 -1 -1 | ||
| 1 -1 | ||
|
|
||
| 0 -1 | ||
| -1 -1 | ||
| ``` | ||
|
|
||
| Tensor and vector aware API can then load the index lookup by loading the dataset as a tensor and access the first positive coordinate at each axis according to the API's axes indexing scheme. | ||
|
|
||
| This creates a named permutation for vectors. | ||
|
|
||
| The ImgLib2 bindings offer API methods to create and use such lookups. Naturally, the mapping is defined as coming from ImgLib2 F-order: | ||
|
|
||
| ```java | ||
|
There was a problem hiding this comment. I could not find $ ag 'createAxes'
doc/axes.md
40:createAxes(
$ ag '(read|write).*Vect'
doc/axes.md
44:readDoubleVectorAttribute(
49:writeDoubleVectorAttribute(
55:readLongVectorAttribute(
60:writeLongVectorAttribute(There was a problem hiding this comment. It seems that the actual naming is a bit different, and they are called |
||
| createAxes( | ||
| String axesName, | ||
| int[] axes); | ||
|
|
||
| readDoubleVectorAttribute( | ||
| N5Reader n5, | ||
| String groupName, | ||
| String attributeName, | ||
| int[] axes); | ||
| writeDoubleVectorAttribute( | ||
| double[] vector, | ||
| N5Writer n5, | ||
| String groupName, | ||
| String attributeName, | ||
| int[] axes); | ||
| readLongVectorAttribute( | ||
| N5Reader n5, | ||
| String groupName, | ||
| String attributeName, | ||
| int[] axes); | ||
| writeLongVectorAttribute( | ||
| double[] vector, | ||
| N5Writer n5, | ||
| String groupName, | ||
| String attributeName, | ||
| int[] axes); | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really like that
.axesis in the root of the container. This makes datasets less portable (i.e. I can't just copy the dataset itself but I will have to remember to also copy the appropriate dataset in.axes. Also, what if a dataset with the same name already exists in.axesof the target container?We could also store
.axesinside the actual dataset, e.g.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
then we cannot use the dataset loader to resolve the ordering because there cannot be a dataset in a dataset (e.g. in HDF5). I agree about it making portability harder which is only an issue if this feature is used which it currently isn't.