HDF5.jl icon indicating copy to clipboard operation
HDF5.jl copied to clipboard

Inconsistent writing of complex data inside compound type

Open olof3 opened this issue 1 year ago • 1 comments

Complex numbers are written differently if they are inside a compound type compared to pure complex data.

complex_data = [1+1im, 2+1im]
namedtuple_data = [(z=z, index=k) for (k,z) in enumerate(complex_data)]

println("Original complex_data: ", complex_data)
println("Original namedtuple_data: ", namedtuple_data)

h5open("test.h5", "w") do file
    file["complex_data"] = complex_data
    file["namedtuple_data"] = namedtuple_data
end

h5open("test.h5", "r") do file
    println("Written & read complex_data: ", read(file["complex_data"]))
    println("Written & read namedtuple_data: ", read(file["namedtuple_data"]))
end

gives

Original complex_data: Complex{Int64}[1 + 1im, 2 + 1im]
Original namedtuple_data: NamedTuple{(:z, :index), Tuple{Complex{Int64}, Int64}}[(z = 1 + 1im, index = 1), (z = 2 + 1im, index = 2)]

Written & read complex_data: Complex{Int64}[1 + 1im, 2 + 1im]
Written & read namedtuple_data: NamedTuple{(:z, :index), Tuple{NamedTuple{(:re, :im), Tuple{Int64, Int64}}, Int64}}[(z = (re = 1, im = 1), index = 1), (z = (re = 2, im = 1), index = 2)]

With

HDF5.set_complex_field_names("re", "im")

the data that is read back has the same format as what was written,

Original complex_data: Complex{Int64}[1 + 1im, 2 + 1im]
Original namedtuple_data: NamedTuple{(:z, :index), Tuple{Complex{Int64}, Int64}}[(z = 1 + 1im, index = 1), (z = 2 + 1im, index = 2)]

Written & read complex_data: Complex{Int64}[1 + 1im, 2 + 1im]
Written & read namedtuple_data: NamedTuple{(:z, :index), Tuple{Complex{Int64}, Int64}}[(z = 1 + 1im, index = 1), (z = 2 + 1im, index = 2)]

Perhaps this indicates that the inconsistency is due to that the special treatment in #558 (which reads/writes complex numbers in hdf5 files as compound types with fields r and i) is applied to nested complex when they are read but not when they are written?

olof3 avatar Jan 02 '24 17:01 olof3

Indeed, Complex support preceded the mapping between named tuples and compound types. I'm not sure that Iike that we have special treatment of Complex.

My preference here would be for the user to explicitly supply the type as the second parameter to read. This is better in terms of type stability.

mkitti avatar Jan 02 '24 23:01 mkitti