Add support for Float16 by eschnett · Pull Request #1229 · JuliaIO/HDF5.jl

eschnett · 2026-04-27T19:28:02Z

Closes #1228.

mkitti

I think some of these changes might make us dependent on HDF5_jll version 2. Could we detect the HDF5 version before trying to load a possibly non-existent constant?

mkitti · 2026-04-27T19:32:57Z

 """
 function h5p_set_obj_track_times(plist_id, track_times)
    lock(liblock)
-    @show :h5p_set_obj_track_times track_times


I think this is probably unrelated.

mkitti · 2026-04-27T19:53:08Z

+const H5T_NATIVE_UINT32  = _read_const(:H5T_NATIVE_UINT32_g)
+const H5T_NATIVE_INT64   = _read_const(:H5T_NATIVE_INT64_g)
+const H5T_NATIVE_UINT64  = _read_const(:H5T_NATIVE_UINT64_g)
+const H5T_NATIVE_FLOAT16 = _read_const(:H5T_NATIVE_FLOAT16_g)


I think we need something for earlier versions of HDF5 here (pre 2.0).

Copilot

Pull request overview

Adds Float16 (half-float) support to HDF5.jl’s Julia↔HDF5 type conversions to address failures when reading Float16 datasets (Issue #1228), and extends the test suite to cover Float16 and ComplexF16 roundtrips.

Changes:

Add Float16 to the scalar “bits” type set and map HDF5 float types of size 2 bytes to Julia Float16.
Introduce an API constant / type-id mapping for H5T_NATIVE_FLOAT16 to enable writing Float16 datasets.
Extend test/plain.jl to write/read Float16 and ComplexF16 datasets; remove a stray debug @show from an API wrapper.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`test/plain.jl`	Adds Float16 + ComplexF16 read/write test coverage.
`src/types.jl`	Expands `BitsType` union to include `Float16`.
`src/typeconversions.jl`	Adds Float16 HDF5 type-id mapping and Float16 size-based type inference on read.
`src/api/types.jl`	Adds `H5T_NATIVE_FLOAT16` constant binding.
`src/api/functions.jl`	Removes a debug `@show` statement from a generated wrapper.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-27T20:05:39Z

+const H5T_NATIVE_UINT32  = _read_const(:H5T_NATIVE_UINT32_g)
+const H5T_NATIVE_INT64   = _read_const(:H5T_NATIVE_INT64_g)
+const H5T_NATIVE_UINT64  = _read_const(:H5T_NATIVE_UINT64_g)
+const H5T_NATIVE_FLOAT16 = _read_const(:H5T_NATIVE_FLOAT16_g)


H5T_NATIVE_FLOAT16 is loaded unconditionally via _read_const(:H5T_NATIVE_FLOAT16_g). Since _read_const uses dlsym(...; throw_error=true) this will raise during module load if the linked libhdf5 (or the shim exporting *_g symbols) doesn’t provide that symbol. Please gate this with _has_symbol(:H5T_NATIVE_FLOAT16_g) (or a libversion check) and either provide a fallback type id or throw a clear runtime error when Float16 writes are attempted on unsupported HDF5 builds.

Suggested change

const H5T_NATIVE_FLOAT16 = _read_const(:H5T_NATIVE_FLOAT16_g)

const H5T_NATIVE_FLOAT16 = _has_symbol(:H5T_NATIVE_FLOAT16_g) ? _read_const(:H5T_NATIVE_FLOAT16_g) : hid_t(-1)

Copilot · 2026-04-27T20:05:40Z

 hdf5_type_id(::Type{UInt32})    = API.H5T_NATIVE_UINT32
 hdf5_type_id(::Type{Int64})     = API.H5T_NATIVE_INT64
 hdf5_type_id(::Type{UInt64})    = API.H5T_NATIVE_UINT64
+hdf5_type_id(::Type{Float16})   = API.H5T_NATIVE_FLOAT16


hdf5_type_id(::Type{Float16}) = API.H5T_NATIVE_FLOAT16 will fail (at compile time or runtime) if the API constant can’t be loaded on some HDF5 versions/builds. Consider defining this method conditionally (based on API._has_symbol(:H5T_NATIVE_FLOAT16_g) / libversion) and raising a targeted error message when Float16 datatypes are not supported by the linked HDF5 library, or mapping to an available standard half-float type when present.

Suggested change

hdf5_type_id(::Type{Float16}) = API.H5T_NATIVE_FLOAT16

function hdf5_type_id(::Type{Float16})

if API._has_symbol(:H5T_NATIVE_FLOAT16_g)

return API.H5T_NATIVE_FLOAT16

end

throw(ArgumentError("Float16 HDF5 datatypes are not supported by the linked HDF5 library"))

end

Copilot · 2026-04-27T20:05:40Z

+    Af16 = read(fr, "Afloat16")
+    @test convert(Matrix{Float16}, A) == Af16
+    @test eltype(Af16) == Float16


These Float16 read/write assertions are unconditional. If the linked HDF5 build doesn’t support half-float (e.g., older libhdf5 versions still allowed by this package’s compat), the tests may fail or even error during package load due to missing type IDs. Please guard this block behind a capability check (e.g., API._has_symbol(:H5T_NATIVE_FLOAT16_g) / a libversion check), or bump the minimum supported HDF5_jll/libhdf5 version accordingly.

Suggested change

Af16 = read(fr, "Afloat16")

@test convert(Matrix{Float16}, A) == Af16

@test eltype(Af16) == Float16

if HDF5.API._has_symbol(:H5T_NATIVE_FLOAT16_g)

Af16 = read(fr, "Afloat16")

@test convert(Matrix{Float16}, A) == Af16

@test eltype(Af16) == Float16

end

Copilot · 2026-04-27T20:05:41Z

+    Acmplx16 = read(fr, "Acmplx16")
+    @test convert(Matrix{ComplexF16}, Acmplx) == Acmplx16
+    @test eltype(Acmplx16) == ComplexF16


ComplexF16 datasets are now written/read unconditionally. This will break on HDF5 builds that don’t support Float16 type IDs (since complex compound type creation relies on hdf5_type_id(Float16)). Please make these tests conditional on Float16 datatype support (or update the minimum supported HDF5 version) to avoid failures on older libhdf5 installations.

eschnett · 2026-04-27T23:50:02Z

Which HDF5 library versions should we support? I might be a good idea to add respective CI tests. It would be straightforward to e.g. run a test with HDF5_jll @1 to test with the latest HDF5 1.x version.

mkitti · 2026-04-28T00:19:46Z

Based on the available downloads I would say 1.14.6 and 2.1.0 at least. Maybe a 1.12 release would be useful as well if there is not a strong need to drop support.

Let me know if you start working on that. I will take a look.

https://support.hdfgroup.org/downloads/

mkitti · 2026-04-28T00:58:02Z

We can drop HDF5 1.12. Thanks for trying.

mkitti · 2026-04-28T01:00:20Z

Or just use Julia 1.10 with HDF5 1.12.

https://github.com/JuliaRegistries/General/blob/master/jll%2FH%2FHDF5_jll%2FCompat.toml#L11-L12

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-28T01:15:49Z

+      - name: Run ${{ matrix.package.repo }} tests with HDF5_jll ${{ matrix.hdf5_jll }}
        env:
          JULIA_DEBUG: Main
+          HDF5_jll: ${{ matrix.hdf5_jll }}
+        shell: julia --project=@. {0}


matrix.package.repo is not defined in the HDF5 job matrix, so this step name will expand to an empty string (and is confusing to read). Use a literal package name (e.g. HDF5.jl) or ${{ github.repository }}/${{ github.workflow }} instead.

Good point.

Copilot · 2026-04-28T01:15:49Z

-          - {version: 'nightly', os: macOS-26, arch: aarch64}
-          - {version: 'nightly', os: windows-2025, arch: x64}
-          - {version: 'nightly', os: windows-2025, arch: x86}
+          - {version: '1.10', os: ubuntu-24.04, arch: x64, hdf5_jll: '1.12'} # HDF5_jll 1.12 does not work with Julia 1.12


The inline comment mentions “Julia 1.12”, but this matrix entry is for Julia 1.10. If the note is still relevant, clarify what combination is known-bad (Julia vs HDF5_jll) to avoid misleading future maintainers.

Suggested change

- {version: '1.10', os: ubuntu-24.04, arch: x64, hdf5_jll: '1.12'} # HDF5_jll 1.12 does not work with Julia 1.12

- {version: '1.10', os: ubuntu-24.04, arch: x64, hdf5_jll: '1.12'} # Keep coverage for Julia 1.10 with HDF5_jll 1.12; the known-bad combination is Julia 1.12 with HDF5_jll 1.12

mkitti · 2026-04-28T01:17:57Z

Looks pretty good to me.

To summarize the support for upstream HDF5_jll:

Support HDF5 1.12 for as long as we support Julia 1.10 LTS
Support last 1.14 release as the last of the HDF5 1.x series, until HDF Group drops that from Downloads page.
Support latest HDF5 2.x release

mkitti · 2026-04-28T01:23:12Z

Both of Copilot's suggestions seem productive with the first one being a bug fix.

Try to keep julia-runtest action. Reformat and elaborate on comments. Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com>

mkitti · 2026-04-28T04:01:15Z

Would we want to add support for bfloat16 in the future perhaps as a package extension via https://github.com/JuliaMath/BFloat16s.jl?
https://www.hdfgroup.org/2025/11/10/release-of-hdf5-2-0-0-newsletter-207/

Add support for Float16

d2d4386

mkitti requested changes Apr 27, 2026

View reviewed changes

mkitti requested a review from Copilot April 27, 2026 20:00

Copilot started reviewing on behalf of mkitti April 27, 2026 20:00 View session

Copilot AI reviewed Apr 27, 2026

View reviewed changes

eschnett added 4 commits April 27, 2026 20:31

Add checks whether the HDF5 library supports Float16

ca677b0

CI: Test HDF5_jll 1.12 and 1.14

1f97e35

Reformat source code

e545374

CI: Correct build script

1a38a1b

CI: Test HDF5_jll 1.12 with Julia 1.10

82ee49b

mkitti requested a review from Copilot April 28, 2026 01:11

Copilot started reviewing on behalf of mkitti April 28, 2026 01:12 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

mkitti reviewed Apr 28, 2026

View reviewed changes

Comment thread .github/workflows/CI.yml Outdated

mkitti reviewed Apr 28, 2026

View reviewed changes

Comment thread .github/workflows/CI.yml Outdated

Address copilot code review

b089094

Try to keep julia-runtest action. Reformat and elaborate on comments. Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com>

mkitti reviewed Apr 28, 2026

View reviewed changes

Comment thread .github/workflows/CI.yml Outdated

Remove second julia-buildpkg

54ae520

mkitti merged commit 350d496 into master Apr 28, 2026
29 checks passed

mkitti deleted the eschnett/float16 branch April 28, 2026 04:05

	const H5T_NATIVE_FLOAT16 = _read_const(:H5T_NATIVE_FLOAT16_g)
	const H5T_NATIVE_FLOAT16 = _has_symbol(:H5T_NATIVE_FLOAT16_g) ? _read_const(:H5T_NATIVE_FLOAT16_g) : hid_t(-1)

-hdf5_type_id(::Type{Float16})   = API.H5T_NATIVE_FLOAT16
+function hdf5_type_id(::Type{Float16})
+    if API._has_symbol(:H5T_NATIVE_FLOAT16_g)
+        return API.H5T_NATIVE_FLOAT16
+    end
+    throw(ArgumentError("Float16 HDF5 datatypes are not supported by the linked HDF5 library"))
+end

	- {version: '1.10', os: ubuntu-24.04, arch: x64, hdf5_jll: '1.12'} # HDF5_jll 1.12 does not work with Julia 1.12
	- {version: '1.10', os: ubuntu-24.04, arch: x64, hdf5_jll: '1.12'} # Keep coverage for Julia 1.10 with HDF5_jll 1.12; the known-bad combination is Julia 1.12 with HDF5_jll 1.12

Conversation

eschnett commented Apr 27, 2026

Uh oh!

mkitti left a comment

Choose a reason for hiding this comment

Uh oh!

mkitti Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

mkitti Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

eschnett commented Apr 27, 2026

Uh oh!

mkitti commented Apr 28, 2026

Uh oh!

mkitti commented Apr 28, 2026

Uh oh!

mkitti commented Apr 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

mkitti Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

mkitti commented Apr 28, 2026

Uh oh!

mkitti commented Apr 28, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mkitti commented Apr 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants