HDF5 Release 2.0.0

🔆 Executive Summary: HDF5 Version 2.0.0

Performance Enhancements:

• Up to 2500% faster Virtual Dataset read/write operations
• 30% faster opening and 25% faster closing of virtual datasets.
• Reduced memory overhead via shared name strings and optimized spatial search algorithms for virtual datasets.

Significant Advancements:

• Full UTF-8 filename support on Windows, resolving encoding issues from previous versions.
• Introduction of bfloat16 predefined datatypes for efficient machine learning conversions.
• First-class support for complex numbers, eliminating manual workarounds in scientific applications.
• A new, larger chunk size limit, in multi-petabytes, replaces the previous 4 GiB limit.

Updated Foundation:

• New file format version (4.0) and compliance with the C11 standard.
• Adopted semantic versioning to clearly convey changes between versions.

Important

• Transitioned to CMake-only builds, and Autotools is no longer in use.
• Renamed library state variables, notably HDF5_ENABLE_PARALLEL is now HDF5_PROVIDES_PARALLEL, see PR #5716 for more details.
• The default setting for H5Fset_libver_bounds has been updated to set the lower bound to the HDF5 library version 1.8. This change ensures that users can take advantage of the library's optimal performance and the latest features by default. If users need their files to be compatible with older versions of the HDF5 library, they will need to adjust this lower bound manually.

Enhanced Features:

• Improved ROS3 VFD capabilities using the aws-c-s3 library.

Java Enhancements:

• Java FFM bindings generated for the C library if Java 25+ available and JNI option is FALSE.
• Enhanced Maven artifact deployment with comprehensive multi-platform support (Linux, Windows, macOS x86_64, macOS aarch64).
• Complete Java examples Maven integration (org.hdfgroup:hdf5-java-examples) with cross-platform CI/CD testing.

Acknowledgements:

We would like to thank the many HDF5 community members who contributed to HDF5 2.0.