Convert Error in dfile$obj_copy_from

[igordot]

I am trying to convert from AnnData to Seurat via h5Seurat. The process started, but then I got an error and an incomplete h5seurat was generated.

> Convert("file.h5ad", dest = "h5seurat", overwrite = TRUE) 
Warning: Unknown file type: h5ad
Warning: 'assay' not set, setting to 'RNA'
Creating h5Seurat file for version 3.1.5.9900
Adding X as data
Adding X as counts
Adding meta.features from var
Error in dfile$obj_copy_from(src_loc = source, src_name = "obs", dst_name = "meta.data") : 
  HDF5-API Errors:
    error #000: H5Ocopy.c in H5Ocopy(): line 233: unable to copy object
        class: HDF5
        major: Object header
        minor: Unable to copy object

    error #001: H5Ocopy.c in H5O__copy(): line 317: unable to copy object
        class: HDF5
        major: Object header
        minor: Unable to copy object

There were a few other error messages. I am just showing the first ones. Any idea how I may resolve this?

[hayfre]

Hi, I am getting the exact same error message when trying to convert from AnnData to Seurat. Any follow-up solutions?

[diazdc]

I'm dealing with the same problem. I thought it was a file permissions error at first, but chmod 777 didn't fix it.

[selifeski]

I am facing the same problem. Very weird because today it worked for another anndata but now I cannot make it worked and it gave the same long error.

[billadelonge]

Getting the same! :(

[jayypaul]

Same!!!

[makloufg]

Same here!

[minhtran1309]

I am getting the same issue It has been already two months since the first time it gets reported. Has anyone solved it? Thanks

[willrosenow]

I am having the same issue

[jloske]

I also got the same error. I created a new env and reinstalled the packages, this time convert worked but when I wanted to use <- LoadH5Seurat() it ran into several problems. I even tried to remove the coloumns causing the issue but it kept running into next errors. Any solutions?

Validating h5Seurat file

Initializing RNA with data

Adding counts for RNA

Adding scale.data for RNA

Adding feature-level metadata for RNA

Error in `[[<-`(`*tmp*`, col.name, value = metadata): [[<- defined for objects of type "S4" only for subclasses of environment
Traceback:

1. LoadH5Seurat("file.h5seurat")
2. LoadH5Seurat.character("file.h5seurat")
3. LoadH5Seurat(file = hfile, assays = assays, reductions = reductions, 
 .     graphs = graphs, neighbors = neighbors, images = images, 
 .     meta.data = meta.data, commands = commands, misc = misc, 
 .     tools = tools, verbose = verbose, ...)
4. LoadH5Seurat.h5Seurat(file = hfile, assays = assays, reductions = reductions, 
 .     graphs = graphs, neighbors = neighbors, images = images, 
 .     meta.data = meta.data, commands = commands, misc = misc, 
 .     tools = tools, verbose = verbose, ...)
5. as.Seurat(x = file, assays = assays, reductions = reductions, 
 .     graphs = graphs, neighbors = neighbors, images = images, 
 .     meta.data = meta.data, commands = commands, misc = misc, 
 .     tools = tools, verbose = verbose, ...)
6. as.Seurat.h5Seurat(x = file, assays = assays, reductions = reductions, 
 .     graphs = graphs, neighbors = neighbors, images = images, 
 .     meta.data = meta.data, commands = commands, misc = misc, 
 .     tools = tools, verbose = verbose, ...)
7. AssembleAssay(assay = assay, file = x, slots = assays[[assay]], 
 .     verbose = verbose)
8. AddMetaData(object = obj, metadata = meta.data)
9. AddMetaData.Assay(object = obj, metadata = meta.data)
10. .AddMetaData(object = object, metadata = metadata, col.name = col.name)

[jayypaul]

Hi all,

I tried looking at my past notes where I got this to work but couldn't find exactly what I did. I remember that it had to do with the metadata sheet, something was wrong with the column names i think.... I suggest starting there! Best o luck

[pcm32]

Also getting this. Using the pbmc3k_final.h5ad from the vignettes. Installed seurat with conda and seurat disk with an in-progress PR to conda forge:

hdf5                      1.10.6          nompi_h34ad4e8_1111    conda-forge
...
r-hdf5r                   1.3.3             r40h4a68bb8_0    conda-forge
...
r-seurat                  4.0.0             r40h4efb63e_0    conda-forge
r-seuratdisk              0.0.9015                      0    local
r-seuratobject            4.0.0             r40h4efb63e_0    conda-forge

Installation of r-seuratdisk happens through R CMD INSTALL --build . on commit 3848eab6fcb9ead489cabdad464921e43561e28c (see https://github.com/conda-forge/staged-recipes/pull/14134).

Any known issues with specific hdf5 library versions that we should avoid? The same SeuratDisk seems to work otherwise (other calls).

[pcm32]

Also happens if I try building with latest commit (743d7397bd48c03575522881d3cded4796ab7006) or when trying with 0.09014 (95d27f636e719cf5fa49b02b05876354343138cd) as suggested in another issue.

[pcm32]

Downgrading r-hdf5r to 1.3.2 seems to have no effect either.

[andrewjkwok]

Was wondering if there are any updates to this as I am getting the exact same problem?

[chris-rands]

Did anyone resolve this? There seem to be multiple related issues here: https://github.com/mojaveazure/seurat-disk/issues/47 https://github.com/mojaveazure/seurat-disk/issues/53 https://github.com/mojaveazure/seurat-disk/issues/64 https://github.com/mojaveazure/seurat-disk/issues/70 and on hdf5r
, https://github.com/hhoeflin/hdf5r/issues/166, and seurat https://github.com/satijalab/seurat/issues/3164

I have this error even when saving the h5ad object with compression="gzip" or "lzf" Would be grateful for any help cc @mojaveazure

[chris-rands]

I think I found a (partial) solution, save with gzip compression and as dense matrix, e.g. python -c "import scanpy as sc; adata = sc.read('pbmc3k_final.h5ad'); adata.write('pbmc3k_final_gzip.h5ad', compression='gzip', force_dense=True)" .

I am also having to change some meta-data attributes, specifically adding adata.raw and deleting the adata.var attributes

[Echo226]

I am getting the same issue.

[flde]

Getting the same error.

[zhanglab2008]

same issue for me

[hayfre]

To avoid this error you can convert from AnnData to loom and then use the loomR package to convert from loom to Seurat. https://satijalab.org/loomr/loomr_tutorial

[MikaQiao]

I am getting the same issue.

[Xenolevis]

I fixed this by removing libhdf5-* version 1.10.x that came with my system (linux) and compiling my own hdf5 API. Latest 1.8 version (1.8.22) seems to work fine. I have also tried 1.10.3 but no luck. Earlier 1.10.x versions are not available from the HDF Group.

[ACastanza]

I am also experiencing this issue.

Full Error Message:

Convert(infile, dest = "h5seurat", overwrite = TRUE)
Warning: Unknown file type: h5ad
Warning: 'assay' not set, setting to 'RNA'
Creating h5Seurat file for version 3.1.5.9900
Adding X as data
Adding X as counts
Adding meta.features from var
Error in dfile$obj_copy_from(src_loc = source, src_name = "obs", dst_name = "meta.data") :
  HDF5-API Errors:
    error #000: H5Ocopy.c in H5Ocopy(): line 233: unable to copy object
        class: HDF5
        major: Object header
        minor: Unable to copy object

    error #001: H5Ocopy.c in H5O__copy(): line 317: unable to copy object
        class: HDF5
        major: Object header
        minor: Unable to copy object

    error #002: H5Ocopy.c in H5O__copy_obj(): line 1221: unable to copy object
        class: HDF5
        major: Object header
        minor: Unable to copy object

    error #003: H5Ocopy.c in H5O__copy_header(): line 1165: unable to copy object
        class: HDF5
        major: Object header
        minor: Unable to copy object

    error #004: H5Ocopy.c in H5O__copy_header_real(): line 872: unable to perform 'post copy' operation on message
        class: HDF5
        major: Object header
        minor: Unable to initialize object

    error #005: H5Ostab.c in H5O__stab_post_copy_file(): line 403: iteration operator failed
        class: HD
Error in private$closeFun(id) : HDF5-API Errors:
    error #000: H5F.c in H5Fclose(): line 675: closing file ID failed
        class: HDF5
        major: File accessibilty
        minor: Unable to close file

    error #001: H5Fint.c in H5F__close(): line 1952: decrementing file ID failed
        class: HDF5
        major: File accessibilty
        minor: Unable to decrement reference count

    error #002: H5I.c in H5I_dec_app_ref(): line 1300: can't decrement ID ref count
        class: HDF5
        major: Object atom
        minor: Unable to decrement reference count

    error #003: H5Fint.c in H5F__close_cb(): line 2010: can't close file
        class: HDF5
        major: File accessibilty
        minor: Unable to close file

    error #004: H5Fint.c in H5F_try_close(): line 2181: problems closing file
        class: HDF5
        major: File accessibilty
        minor: Unable to close file

    error #005: H5Fint.c in H5F__dest(): line 1277: problems closing file
        class: HDF5
        major: File
>

h5ad file was written out from scVelo using AnnData v0.7.6 with compression="gzip". Also tried with force_dense=True and this did not resolve the issue.

Failing h5ad file here: https://drive.google.com/file/d/1Mt6-yiGyWCLC5pwc5AmVMRlA7rbQps1K/view?usp=sharing

[Xenolevis]

@ACastanza I'm able to convert your file on my setup with hdf5 API 1.8.22. What OS are you on?

[ACastanza]

@Xenolevis I am on a Mac (10.15.7), with packages installed via conda.

I'm not entirely sure which package would be providing the HDF5 API you mention, but the HDF5 packages installed on my system are as follows:

bioconductor-hdf5array 1.18.1 r40h68a2ddb_0 bioconda bioconductor-rhdf5 2.34.0 r40h832b226_1 bioconda bioconductor-rhdf5filters 1.2.0 r40h64ad5ed_0 bioconda bioconductor-rhdf5lib 1.12.1 r40h68a2ddb_0 bioconda hdf5 1.10.6 nompi_h34ad4e8_1111 conda-forge r-hdf5r 1.3.3 r40h4a68bb8_0 conda-forge

[Xenolevis]

@ACastanza it's the hdf5 one. None of the available 1.10.x. or 1.12.x that I have tested work for me. I see 1.8.20 is available from conda-forge, and version 1.8.22 from Homebrew. Unfortunately I don't quite speak macOS, so I can't offer better directions. Basically you would have to:

1. remove the R packages Seurat-disk and hdf5r.
2. downgrade the hdf5 library.
3. reinstall the R packages.

Good luck!

[ACastanza]

I see, I'll give it a shot. Or just create a new conda environment with that version.

It does sound like this is a bug in seurat-disk then; that it needs to be updated to support some change in the api, and until then, the package manifests need to explicitly specify hdf5<.10.6 so that conda can install a working version when it resolves the dependency conflicts.

[Xenolevis]

At least equally likely to be an issue of hdf5r. It's known to be compatible only with version up to 1.10.0 of the hdf5 api, according to its readme.

[ACastanza]

Yeah that seems to be a problem with hdf5r as well, in it's package description it's specifying

SystemRequirements: HDF5 (>= 1.8.13) https://github.com/hhoeflin/hdf5r/blob/master/DESCRIPTION

which is allowing it to install the incompatible 1.10.x versions.

So something definitely changed in the latest releases that these packages need to account for.

[ACastanza]

Even a very basic anaconda environment can't seem to resolve the dependencies necessary to get this older version of hdf5 to install. This is definitely something that the seurat-disk and hdf5r folks are going to need to resolve.

Edit: the fundamental incompatibiltiy seems to be down to the libfortran version required, it's impossible to install R 4.x alongside this old version of the hdf5 library:

r-base=4 -> libgfortran[version='5.*|>=4.0.0,<5.0.0.a0'] hdf5=1.8.20 -> libgfortran[version='>=3.0.1,<4.0.0.a0']