Skip to content

Conversation

brtnfld
Copy link
Collaborator

@brtnfld brtnfld commented Sep 15, 2025

Maps logical file to physical subfiles: When HDF5 uses the subfiling VFD, a single logical HDF5 file is actually split across multiple physical subfiles stored on different nodes. The new API returns subfile paths, providing the complete list of subfile names/paths that make up the logical file for a given MPI rank. This enables file reconstruction by using tools like h5fuse to recombine the subfiles into a single HDF5 file.

Closes #5120


Important

Adds H5FDsubfiling_get_file_mapping() API to map logical HDF5 files to physical subfiles, with Fortran support and comprehensive testing.

  • New API:
    • Adds H5FDsubfiling_get_file_mapping() in H5FDsubfiling.c to retrieve subfile paths for a logical HDF5 file.
    • Updates Fortran interface in H5VFDff.F90 to include h5fdsubfiling_get_file_mapping_f().
  • Testing:
    • Adds tests in t_subfiling_vfd.c for H5FDsubfiling_get_file_mapping() covering various scenarios including error conditions and I/O operations.
  • Utilities:
    • Updates h5fuse.in to support a list of subfiles for processing with a new -l option.

This description was created by Ellipsis for 9501371. You can customize this summary. It will automatically update as commits are pushed.

@brtnfld brtnfld requested a review from mattjala as a code owner September 15, 2025 16:28
@github-project-automation github-project-automation bot moved this to To be triaged in HDF5 - TRIAGE & TRACK Sep 15, 2025
@brtnfld brtnfld marked this pull request as draft September 15, 2025 16:28
@brtnfld brtnfld added Component - C Library Core C library issues (usually in the src directory) Component - Parallel Parallel HDF5 (NOT thread-safety) HDFG-internal Internally coded for use by the HDF Group labels Sep 15, 2025
@brtnfld brtnfld marked this pull request as ready for review September 17, 2025 19:29
@brtnfld brtnfld requested a review from epourmal as a code owner September 17, 2025 19:29
! **** _| |_| | | | | | |__| | | \ \ | |/ ____ \| |\ | | | ****
! |_____|_| |_|_| \____/|_| \_\ |_/_/ \_\_| \_| |_|
!
! If you add a new H5VL function you must add the function name to the
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo noticed in the documentation: "H5VL function" appears instead of "H5VFD function". Considering this module provides the Fortran interface for H5VFD, please update the comment accordingly.

Suggested change
! If you add a new H5VL function you must add the function name to the
! If you add a new H5VFD function you must add the function name to the

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component - C Library Core C library issues (usually in the src directory) Component - Parallel Parallel HDF5 (NOT thread-safety) HDFG-internal Internally coded for use by the HDF Group
Projects
Status: To be triaged
Development

Successfully merging this pull request may close these issues.

Add a subfiling API call that returns the subfile/rank/node mapping
2 participants