Skip to content

HDF5 2.0 Planning

Jump to bottom
Dana Robinson edited this page Oct 22, 2024 · 11 revisions

These are the topics/issues we're going to focus on in the next version of HDF5. This list is ambitious, so not everything here is realistically going to make it in.

NOTE: This list is being adapted from an earlier HDF5 1.16.0 to-do list. It should be finalized by early November (for a March 2025 release).

Tentative 2.0.0 changes

  • Update library defaults to provide better performance with cloud-optimized HDF5 and modern I/O hardware
  • Complex number support (requires bumping an internal version number)
  • Build system changes
    • Better alignment between Autotools and CMake
    • Might drop/rename some build features
  • Use HDoff_t instead of off_t in public API

Bugfixes and general quality

  • Fix all oss-fuzz and other segfault-related bugs (there are several in Jira) and harden surrounding code
  • Tackle bugs in GitHub issue backlog, shooting for < 50 unfixed at release time
  • -fsanitize=memory/address/undefined clean and added to CI
  • Remove Mercury code from subfiling
  • Fix file image bugs and review code (#1915)

Cloud-Optimized HDF5

  • Review and possibly refactor ros3 VFD code
  • Performance testing

Extend warning-free zone

  • Bring suppression scheme to Windows (MSVC and clang), Intel, etc. as needed
  • Look into HPC systems that are not as well-tested
  • Look into compilers that are not as well-tested

Support for new datatypes

  • Complex numbers (see above for 1.16.0 notes)
  • ML types (BFLOAT16, FP8, FP4)
  • Boolean
  • GNU int128
  • Fix outstanding long double issues (mainly POWER)

Configure issues

  • Autotools pkg-config support
  • Full Autotools and CMake support for MinGW and Cygwin, including cross-compiling
  • Check in generated source files

Performance

  • Additional subfiling performance profiling and fixing

Windows as a first-class citizen

  • Win32 VFD
  • Unicode/code page file name tests
  • H5Dwrite performance issue
  • Core VFD slowdown

CI/CD/Testing

  • Determine a final version naming/numbering scheme that works well with package management systems
  • Clean up CDash, public CDash all green
  • Schedule a pre-release and code freeze
  • Release signed msi and dmg files (however we have to make that happen)
  • Develop a manual test process
  • Move VOL tool testing to main repo

Documentation

  • All “developer-level” public headers have full Doxygen (VOL, VFD, etc.)
  • Develop a list of user guide improvements that need to be made

Misc

  • Close PR #1387 (minor datatype optimization)
  • Go over plugin, etc. paths and make sure that everything is handled well and uniformly
  • Add missing H5R tests

Tools

  • ph5diff rewrite
  • H5L copy hack in h5repack

Fortran wrappers

  • H5R improvements (also C code)
  • Make sure cross-compiling with Fortran works well
Clone this wiki locally