Proposal: Add `AnatomicalCoordinatesImage` Neurodata Type

[alessandratrapani]

Proposal: Add AnatomicalCoordinatesImage Neurodata Type

Summary

Propose a new Neurodata type AnatomicalCoordinatesImage to enable registration of entire fields of view (FOV) into anatomical coordinate spaces, complementing the existing point-based AnatomicalCoordinatesTable.

Proposed Implementation

Container Structure

AnatomicalCoordinatesImage would be an NWBContainer similar to AnatomicalCoordinatesTable, with the following structure:

AnatomicalCoordinatesImage:
  neurodata_type_def: AnatomicalCoordinatesImage
  neurodata_type_inc: NWBContainer
  doc: Anatomical coordinates for an entire field of view or imaging plane
  
  attributes:
      - name: method
        doc: "The method used to determine the coordinates"
        dtype: text
        required: true
  
  links:
    - name: space
       target_type: Space
       quantity: 1
       doc: "The space in which the coordinates are defined"
       name: space
    - name: target
      target_type: [ImagingPlane, Image]
      doc: Reference to the ImagingPlane or Image object this registration applies to
      required: true
  
  datasets:
    - name: data
      dtype: float32
      dims:
        - - width
          - height
          - axis
      shape:
        - - null
          - null
          - 3
      doc: 3D array containing XYZ coordinates for each pixel (width x height x 3)
      required: true
    
    - name: brain_region
      dtype: text
      dims:
        - - width
          - height
      shape:
        - - null
          - null
      doc: 2D array of brain region names for each pixel
      required: false
    
    - name: brain_region_id
      dtype: int32
      dims:
        - - width
          - height
      shape:
        - - null
          - null
      doc: 2D array of brain region IDs for each pixel (corresponding to atlas ontology)
      required: false

Key Fields

1. data (required): 3D matrix of shape (width, height, 3) containing the XYZ anatomical coordinates for each pixel

   • Dimension order follows standard image conventions (width, height, coordinates)
   • The third dimension contains [X, Y, Z] coordinates in the specified reference frame

2. brain_region (optional): 2D matrix of shape (width, height) storing brain region string labels

   • Allows direct lookup of anatomical region name for each pixel
   • Useful for visualization and region-based analysis

3. brain_region_id (optional): 2D matrix of shape (width, height) storing brain region ontology IDs

   • Enables programmatic queries and hierarchical anatomical analysis
   • IDs should correspond to a standard ontology (e.g., Allen Brain Atlas)

4. target (required link): Points to either an ImagingPlane or Image object

   • Explicitly links the registration to the imaging data it describes
   • Supports both timeseries imaging (through the ImagingPlane object) and static images, e.g. summary images as MeanImage, MaximumProjection etc.

Usage Example

from pynwb import NWBFile, NWBHDF5IO
from pynwb.ophys import ImagingPlane
import numpy as np

nwbfile = mock_NWBFile()

localization = Localization()
nwbfile.add_lab_meta_data([localization])

space = Space.get_predefined_space("CCFv3")
localization.add_spaces([space])

# Create imaging plane
imaging_plane = ImagingPlane(
    name="imaging_plane",
    description="Primary visual cortex imaging",
    device=device,
    excitation_lambda=920.0,
    imaging_rate=30.0,
    indicator="GCaMP6f",
    location="V1",
    optical_channel=optical_channel
)

# Create registration data
width, height = 512, 512
coordinates = np.random.randn(width, height, 3)  # XYZ coords for each pixel
region_ids = np.zeros((width, height), dtype=np.int32)
region_names = np.full((width, height), "VISp", dtype=object)

# Create AnatomicalCoordinatesImage
anatomical_coords_image = AnatomicalCoordinatesImage(
    name="fov_registration",
    description="Registration of FOV to Allen CCFv3",
    target=imaging_plane,
    reference_frame="Allen CCFv3",
    units="micrometers",
    data=coordinates,
    brain_region=region_names,
    brain_region_id=region_ids
)

# Add to NWB file
localization.add_anatomical_coordinates_image([anatomical_coords_image])

Relationship to Existing Types

This proposal complements AnatomicalCoordinatesTable:

• AnatomicalCoordinatesTable: Sparse, ROI-based registration (good for segmented cells/regions)
• AnatomicalCoordinatesImage: Dense, pixel-based registration (good for full FOV mapping)

Both can coexist in the same NWB file for different purposes.

Questions for Discussion

1. Should we support Volumetric Imaging Data with another Neurodata type, e.g., AnatomicalCoordinatesVolume?
2. Should there be a quality/confidence metric associated with each pixel's registration?
3. Would it be useful to include transformation parameters (affine matrix, warp fields) in addition to final coordinates?

Implementation Checklist

If approved, implementation would include:

• Update schema YAML definition
• Generate PyNWB classes
• Add validation tests
• Create example notebooks
• Update documentation
• Add integration tests with ImagingPlane and Image objects

[alessandratrapani]

Alternative for data

  datasets:
    - name: x
      dtype: float32
      dims:
        - - width
          - height
      shape:
        - - null
          - null
      doc: 2D array containing X coordinates for each pixel (width x height)
      required: true
    - name: y
      dtype: float32
      dims:
        - - width
          - height
      shape:
        - - null
          - null
      doc: 2D array containing Y coordinates for each pixel (width x height)
      required: true
    - name: z
      dtype: float32
      dims:
        - - width
          - height
      shape:
        - - null
          - null
      doc: 2D array containing Z coordinates for each pixel (width x height)
      required: true

Volumetric case:

  datasets:
    - name: x
      dtype: float32
      dims:
        - - width
          - height
          - depth
      shape:
        - - null
          - null
          - null
      doc: 3D array containing X coordinates for each pixel (width x height x depth )
      required: true
    - name: y
      dtype: float32
      dims:
        - - width
          - height
          - depth
      shape:
        - - null
          - null
          - null
      doc: 3D array containing Y coordinates for each pixel (width x heightx depth )
      required: true
    - name: z
      dtype: float32
      dims:
        - - width
          - height
          - depth
      shape:
        - - null
          - null
          - null
      doc: 3D array containing Z coordinates for each pixel (width x heightx depth )
      required: true