ERROR: LoadError: Error closing object

[cndesantana]

We are running a program to parse many ascii files. We want to identify the keywords of such ascii files and to compare the keywords of some files with the keywords of other files. We are using JLD.jl to write the intermediate results into .jld files.

We use to compare hundreds of files among each other. Sometimes the program crashes and returns the following error message:

HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 139869298599808: #000: ../../../src/H5O.c line 1059 in H5Oclose(): not a valid object major: Invalid arguments to routine minor: Bad value ERROR: LoadError: Error closing object in h5o_close at /home/cdesantana/.julia/v0.4/HDF5/src/plain.jl:1927 in close at /home/cdesantana/.julia/v0.4/JLD/src/JLD.jl:122 in jldopen at /home/cdesantana/.julia/v0.4/JLD/src/JLD.jl:242 in load at /home/cdesantana/.julia/v0.4/JLD/src/JLD.jl:1180 in load at /home/cdesantana/.julia/v0.4/FileIO/src/loadsave.jl:42 in load_pairs_repeats at /home/cdesantana/Data/where2publish/src/creategraph.jl:120 in text_graph at /home/cdesantana/Data/where2publish/src/creategraph.jl:145 in anonymous at /home/cdesantana/Data/where2publish/src/index.jl:56 in open at iostream.jl:114 in compare_manyabstracts! at /home/cdesantana/Data/where2publish/src/index.jl:45 in main at /home/cdesantana/Data/where2publish/src/index.jl:78 in include at ./boot.jl:261 in include_from_node1 at ./loading.jl:304 in process_options at ./client.jl:308 in _start at ./client.jl:411

The line of our code where the program crashes is the following one:

pairs = load(file_name, "pairs")

Something curious is that after the program crashes we are able to run it again starting from the file for which it crashed. So I suppose this error may be related to an overload of the JLD.jl functions we use or something like that.

Any idea?

[nalimilan]

Thanks for the report. Do you think you would be able to provide a script plus some (real or fake) data which reproduces the problem? Maybe loading repeatedly the same file is equivalent to loading lots a different ones?

[cndesantana]

Hi,

Thanks for the quick response. I will test if I can reproduce the error by loading only one or a few files repeatedly. If I can reproduce it I will then share with you the code and the input files.

I hope to write you back soon!

Best,

---

From: Milan Bouchet-Valat [[email protected]] Sent: Friday, November 13, 2015 1:16 PM To: JuliaLang/JLD.jl Cc: De Santana, Charles Subject: Re: [JLD.jl] ERROR: LoadError: Error closing object (#41)

Thanks for the report. Do you think you would be able to provide a script plus some (real or fake) data which reproduces the problem? Maybe loading repeatedly the same file is equivalent to loading lots a different ones?

— Reply to this email directly or view it on GitHubhttps://github.com/JuliaLang/JLD.jl/issues/41#issuecomment-156416507.

[crbinz]

Hello,

I think I'm running into a similar issue while using JPLEphemeris.jl. In particular, getephem() parses a bunch of ASCII files and uses jldopen to write the parsed data to .jld files.

The time at which I get the error seems to vary, as does the particular set of error messages, but without wrapping the offending jldopen lines in try/catch statements, the error is:

ERROR: unlink: resource busy or locked (EBUSY)
 in unlink at fs.jl:102

followed by (when quitting julia)

HDF5-DIAG: Error detected in HDF5 (1.8.13) thread 0:
  #000: /home/abuild/rpmbuild/BUILD/hdf5-1.8.13/src/H5O.c line 1078 in H5Oclose(): not a valid file object ID (dataset, group, or datatype)
    major: Invalid arguments to routine
    minor: Unable to release object

Wrapping the jldopen calls in try/catch is what led me here, since it prints:

ErrorException("Error closing object")

To reproduce, building JPLEphemeris.jl for the first time should call getephem(430), but if it doesn't, just do

using JPLEphemeris
getephem(430)

[mweastwood]

I periodically run into this error once a month or so. It is not deterministic so re-running my script usually works. Is there a debugging statement I can add to my JLD.jl that will provide useful diagnostic information the next time this happens to me?

[garrison]

Here's a test case that reliably crashes for me using JLD v0.6.7 ~~(which includes the race condition fix #346)~~ and julia 0.5.1-pre+31:

import JLD

function main()
    fn1 = "a.jld"
    fn2 = "b.jld"

    JLD.save(fn1, "data", rand(256))
    JLD.save(fn2, "data", rand(256))

    i = 0
    while true
        JLD.load(fn1, "data")
        JLD.load(fn2, "data")
        i += 1
        println("loop complete ($i times)")
    end
end

main()

Most often this program will crash for me after 622 iterations, but I've also seen it crash after 4443 iterations on multiple occasions, as well as a few other numbers on at least one occasion.

EDIT: And if I comment out the JLD.save lines after running it once, it often crashes after 4193 iterations.

[tkelman]

~~> using JLD v0.6.7 (which includes the race condition fix #346)~~

~~No it doesn't? https://github.com/JuliaIO/HDF5.jl/commit/9bdaab3d7d5f31adb5c7b1f63f7b22f3f2a4aa1e, the v0.6.7 tag was the preceding commit~~

ignore me, looking at wrong repo

[garrison]

@tkelman Turns out, I was looking at the wrong repo too. When I upgrade HDF5.jl to 0.7.1, my test case no longer crashes.