Compatibilty of jhdf created files with H5 command line tools

[ebremer]

Describe the bug

I created a H5 dataset of random byte[] using jhdf. I can view this dataset with HDFView just fine. However, if I try to repack the file with h5repack, it throws an error "Error occurred while repacking" If I use HDFView to create a similar H5 file, even import the same random data into the HDFView-created H5 file, the repacking works.

To Reproduce

I created my file with jhdf with

        byte[] bytes = new byte[100];   
        SecureRandom sec = new SecureRandom();
        sec.nextBytes(bytes);
        ByteBuffer buffer = ByteBuffer.wrap(bytes);
        buffer.order(ByteOrder.LITTLE_ENDIAN);
        try (WritableHdfFile hdfFile = HdfFile.write(Paths.get("/data/j1.h5"))) {
            hdfFile.putDataset("randomdata", bytes);
        } catch (Exception e) {
            e.printStackTrace();
        }

Expected behaviour

That I would be able to repack the file just like the HDFView created file.

Please complete the following information:

• jhdf version v0.9.4
• Java version 24.0.1
• OS (Windows, Mac, Linux) - Windows 11
• Stack trace if available - NONE
• jhdf logging if available - NONE

Attached zip contains j1.h5 (created by jhdf) and h1.h5 created by HDFView with data imported from j1.h5

h5repack commands

h5repack -f GZIP=7 -l CHUNK=1000 j1.h5 j1X.h5
Error occurred while repacking
h5repack -f GZIP=7 -l CHUNK=1000 h1.h5 h1X.h5

second command generates no error twofiles.zip

[jamesmudd]

Thanks for opening this issue. I will need to take a look at the files, but my guess is probably because jHDF does not write the free space index. jHDF is actually packing the file very efficiently in the current implementation but this comes at the cost of no in place editing.

[ebremer]

Thanks @jamesmudd, I was exploring using h5repack to create chunked compressed HDF5 files from ones created by jHDF.....at least until jHDF could do it directly.