HDF5.jl icon indicating copy to clipboard operation
HDF5.jl copied to clipboard

Wrong CSET cause failure to write to already-created attribute

Open JiangXL opened this issue 3 years ago • 3 comments

When I try to create attribute in HDF5.Datatype(HDF5.H5T_C_S1) following this document:

creates groups, datasets, and attributes without writing any data to them. You can then use write(obj, data) to store the data. The optional property lists allow even more fine-grained control. This syntax uses data to infer the object's "HDF5.datatype" and "HDF5.dataspace"; for the most explicit control, data can be replaced with dtype, dspace, where dtype is an HDF5.Datatype and dspace is an HDF5.Dataspace.

key = "test"
value  = "test" 
dspace = dataspace((length((value), ))
attr = create_attribute(parent, key, HDF5.Datatype(HDF5.H5T_C_S1), dspace)
write(attr, value)

Julia will fail to write attribute.

HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
  #000: H5A.c line 684 in H5Awrite(): unable to write attribute
    major: Attribute
    minor: Write failed
  #001: H5VLcallback.c line 1334 in H5VL_attr_write(): write failed
    major: Virtual Object Layer
    minor: Write failed
  #002: H5VLcallback.c line 1301 in H5VL__attr_write(): write failed
    major: Virtual Object Layer
    minor: Write failed
  #003: H5VLnative_attr.c line 206 in H5VL__native_attr_write(): unable to write attribute
    major: Attribute
    minor: Write failed
  #004: H5Aint.c line 736 in H5A__write(): unable to convert between src and dst datatypes
    major: Attribute
    minor: Feature is unsupported
  #005: H5T.c line 4815 in H5T_path_find(): can't find datatype conversion path
    major: Datatype
    minor: Can't get value
  #006: H5T.c line 5028 in H5T__path_find_real(): no appropriate function for conversion path
    major: Datatype
    minor: Unable to initialize object
ERROR: Error writing attribute data

According to

# Write to already-created objects
function Base.write(obj::Attribute, x)
    dtype = datatype(x)
    try
        write_attribute(obj, dtype, x)
    finally
        close(dtype)
    end
end

write(attr, x) will get datatype from x instead of use dtype from created attribute. But datatype(value) is different to HDF5.Datatype(HDF5.H5T_C_S1).

julia> datatype("test")
HDF5.Datatype: H5T_STRING {
      STRSIZE 4;
      STRPAD H5T_STR_NULLTERM;
      CSET H5T_CSET_UTF8;
      CTYPE H5T_C_S1;
   }
julia> HDF5.Datatype(HDF5.H5T_C_S1)
HDF5.Datatype: H5T_STRING {
      STRSIZE 1;
      STRPAD H5T_STR_NULLTERM;
      CSET H5T_CSET_ASCII;
      CTYPE H5T_C_S1;
   }

I want to take this string be ASCIIString, but datatype think it is UTF8String. Now, Julia has already removed ASCIIString. Finally I can write attribute be ASCIIString as below.

attr = create_attribute(parent, key, HDF5.Datatype(HDF5.H5T_C_S1), dspace)
write_attribute(attr, HDF5.Datatype(HDF5.H5T_C_S1), value)

I spend half of day to figure out. I post here to help someone with similar issue.

JiangXL avatar Mar 01 '21 14:03 JiangXL

Thanks for reporting issues you have had. We should defintely document them.

musm avatar Apr 12 '21 19:04 musm

So basically the bug is that write needs to read the attributes from obj ?

It sounds reasonable that the attribute should be inherited.

musm avatar Apr 13 '21 02:04 musm

So basically the bug is that write needs to read the attributes from obj ?

It sounds reasonable that the attribute should be inherited.

I think it will better to be inherited. In my case, if write get Datatype from new assigned value, I will not able to assign a CSET H5T_CSET_ASCII value directly due to Julia remove ACSIIString .

JiangXL avatar Apr 13 '21 04:04 JiangXL