CodecZLib build fails on Linux and WSL

[Crown421]

Dear Team,

I am trying to setup a new server with Julia, and while trying to build JLD2 the build of CodecZlib fails. The error message is the following:

┌ Error: Error building `CodecZlib`:
│ ERROR: LoadError: LibraryProduct(nothing, ["libz"], :libz, "Prefix(/homes/sridderb/.julia/packages/CodecZlib/9jDi1/deps/usr)")
is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] #write_deps_file#165(::Bool, ::Bool, ::Function, ::String, ::Array{LibraryProduct,1}) at /homes/sridderb/.julia/packages/B
inaryProvider/A0sDa/src/Products.jl:419
│  [3] (::getfield(BinaryProvider, Symbol("#kw##write_deps_file")))(::NamedTuple{(:verbose,),Tuple{Bool}}, ::typeof(write_deps_fi
le), ::String, ::Array{LibraryProduct,1}) at ./none:0
│  [4] top-level scope at /homes/sridderb/.julia/packages/CodecZlib/9jDi1/deps/build.jl:93
│  [5] include at ./boot.jl:326 [inlined]
│  [6] include_relative(::Module, ::String) at ./loading.jl:1038
│  [7] include(::Module, ::String) at ./sysimg.jl:29
│  [8] include(::String) at ./client.jl:403
│  [9] top-level scope at none:0
    end
│ in expression starting at /homes/sridderb/.julia/packages/CodecZlib/9jDi1/deps/build.jl:78
│ LibraryProduct[LibraryProduct(nothing, ["libz"], :libz, "Prefix(/homes/sridderb/.julia/packages/CodecZlib/9jDi1/deps/usr)")]

Digging through the source code of CodecZLib's build.jl I find that on line 7 they define

products = [
    LibraryProduct(prefix, ["libz"], :libz),
]

which is at odds with the error above, where appear to be 4 inputs, and the prefix is at the end. Further in line 76

unsatisfied = any(!satisfied(p; verbose=verbose) for p in products)

clearly ends with unsatisfied = false, as we progress to line 96,

write_deps_file(joinpath(@__DIR__, "deps.jl"), products, verbose=verbose)

Now, in BinaryProvider's product.jl we have at line 419 (as mentioned by the error)

for p in products
        if !satisfied(p; verbose=verbose, isolate=isolate)
            error("$p is not satisfied, cannot generate deps.jl!")
        end
end

which looks almost identical to the line above, which goes through.

Further, on my WSL setup, where I ~~have~~ had working JLD2 and CodecZLib, rebuilding it now fails with the same error as above.

I am not sure what to do about this, as I am not highly proficient in Julia yet, but I would appreciate assistance.

Many thanks.

Update After (painfully) downgrading to version 0.5.4 it goes through without any issue, so there appears to be a bug in the latest release.

[BoundaryValueProblems]

With BinaryProvider v0.5.5, I have the same build problems on all of my machines (macOS, Windows 10, ubuntu/linux), not only in CodecZlib, but also, LibCURL, EzXML, MbedTLS, ZMQ, FreeType. Under v0.5.4, all of them were working without any problem. Please fix this ASAP!! The error messages for those packages are in the following format:

ERROR: LoadError: LibraryProduct(nothing, ["libfreetype"], :libfreetype, "Prefix(/Users/xxx/.julia/packages/FreeType/2dE5w/deps/usr)") is not satisfied, cannot generate deps.jl!

[zgornel]

The error seems to be here https://github.com/JuliaPackaging/BinaryProvider.jl/blob/6779f5408a5ec0047ed581a3ffa02edcf044be00/src/Products.jl#L398

isolate=true triggers the error (at least on my Linux system) and this may be related to https://github.com/JuliaPackaging/BinaryProvider.jl/blob/6779f5408a5ec0047ed581a3ffa02edcf044be00/src/Products.jl#L168 which was introduced in v0.5.5. It looks like a Windows-only fix of some sort that breaks on other systems.

This can be easily fixed I suppose and a new bugfix release done. @hhaensel Should be the most informed on the matter...

[hhaensel]

Hi, could you try to comment out the line 168 and see whether that is the root cause? If so, could you please report the value of dl_path

- dl_esc_path = replace(dl_path, "\\"=>"\\\\")
+ dl_esc_path = dl_path
+ @info(dl_path)

UPDATE: on my Windows 7 system, I obtain:

(v1.1) pkg> build -v JLD2
  Building CodecZlib → `C:\Users\m136270\.julia\packages\CodecZlib\9jDi1\deps\build.log`
 Resolving package versions...
[ Info: dl_esc_path: C:\\Users\\m136270\\.julia\\packages\\CodecZlib\\9jDi1\\deps\\usr\\bin\\libz-1.2.11.dll

whithout modification of dl_esc_path and

@info("dl_esc_path: $(dl_esc_path)")

[zgornel]

Hmm, it is not that The output in both cases is (with/without the replace call):

(v1.1) pkg> build -v JLD2
  Building CodecZlib → `~/.julia/packages/CodecZlib/9jDi1/deps/build.log`
[ Info: /home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/usr/lib/libz.so
[ Info: /home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/usr/lib/libz.so.1
[ Info: /home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/usr/lib/libz.so.1.2.11
ERROR: LoadError: LibraryProduct(nothing, ["libz"], :libz, "Prefix(/home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/usr)") is not satisfied, cannot generate deps.jl!
Stacktrace:
 [1] #write_deps_file#165(::Bool, ::Bool, ::Function, ::String, ::Array{LibraryProduct,1}) at /home/zgornel/.julia/packages/BinaryProvider/A0sDa/src/Products.jl:421
 [2] (::getfield(BinaryProvider, Symbol("#kw##write_deps_file")))(::NamedTuple{(:verbose,),Tuple{Bool}}, ::typeof(write_deps_file), ::String, ::Array{LibraryProduct,1}) at ./none:0
 [3] top-level scope at /home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/build.jl:93
 [4] include(::String) at ./client.jl:403
 [5] top-level scope at none:0
in expression starting at /home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/build.jl:78
┌ Error: Error building `CodecZlib`:
└ @ Pkg.Operations /build/julia/src/julia-1.1.1/usr/share/julia/stdlib/v1.1/Pkg/src/Operations.jl:1075

However, I can confirm that setting isolate=false as default in write_deps_file (L398) works.

Tried the following:

dl_esc_path = replace(dl_path, "\\"=>"\\\\")
if success(`$(Base.julia_cmd()) -e "import Libdl; Libdl.dlopen(\"$(dl_esc_path)\")"`)
    return dl_path
else
    println("failed for $dl_esc_path")
    return dl_path
end

And the output is

v1.1) pkg> build -v JLD2
  Building CodecZlib → `~/.julia/packages/CodecZlib/9jDi1/deps/build.log`
failed for /home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/usr/lib/libz.so

However:

dl_esc_path = "/home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/usr/lib/libz.so"
 success(`$(Base.julia_cmd()) -e "import Libdl; Libdl.dlopen(\"$(dl_esc_path)\")"`)

returns true

[hhaensel]

Ah, then it is probably the usage of the quotes somehow

[hhaensel]

Please execute

`$(Base.julia_cmd()) -e "import Libdl; Libdl.dlopen(\"test\")"`

and tell me the response

[zgornel]

Here it is:

`/usr/bin/julia -Cnative -J/usr/lib/julia/sys.so -g1 -e 'import Libdl; Libdl.dlopen("test")'`

[hhaensel]

Well, I don't have an immediate clue ...

[hhaensel]

Could you provide line 93 of your /home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/build.jl

Well, probably better send the whole file

[hhaensel]

If I remove te replacement "\\"=>"\\\\" on Windows I receive very similar output as you

(v1.1) pkg> build -v JLD2
  Building CodecZlib → `C:\Users\hh\.julia\packages\CodecZlib\9jDi1\deps\build.log`
 Resolving package versions...
[ Info: dl_esc_path: C:\Users\hh\.julia\packages\CodecZlib\9jDi1\deps\usr\bin\libz-1.2.11.dll
[ Info: dl_esc_path: C:\Users\hh\.julia\packages\CodecZlib\9jDi1\deps\usr\bin\libz-1.dll
[ Info: dl_esc_path: C:\Users\hh\.julia\packages\CodecZlib\9jDi1\deps\usr\bin\libz.dll
ERROR: LoadError: LibraryProduct(nothing, ["libz"], :libz, "Prefix(C:\\Users\\hh\\.julia\\packages\\CodecZlib\\9jDi1\\deps\\usr)") is not satisfied, cannot generate deps.jl!
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] #write_deps_file#169(::Bool, ::Bool, ::Function, ::String, ::Array{LibraryProduct,1}) at C:\Users\hh\.julia\dev\BinaryProvider\src\Products.jl:420
 [3] (::getfield(BinaryProvider, Symbol("#kw##write_deps_file")))(::NamedTuple{(:verbose,),Tuple{Bool}}, ::typeof(write_deps_file), ::String, ::Array{LibraryProduct,1}) at .\none:0
 [4] top-level scope at C:\Users\hh\.julia\packages\CodecZlib\9jDi1\deps\build.jl:93
 [5] include at .\boot.jl:326 [inlined]
 [6] include_relative(::Module, ::String) at .\loading.jl:1038
 [7] include(::Module, ::String) at .\sysimg.jl:29
 [8] include(::String) at .\client.jl:403
 [9] top-level scope at none:0
in expression starting at C:\Users\hh\.julia\packages\CodecZlib\9jDi1\deps\build.jl:78
┌ Error: Error building `CodecZlib`:
└ @ Pkg.Operations C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Pkg\src\Operations.jl:1075

So I think it comes from the fact that the system does not find the file. I assume it is a symlink?

[hhaensel]

In that case you could try to replace the symlink with a copy of the original file and retest. Alternatively you could try to create an environment vaiable BINARYPROVIDER_COPYDEREF and set it to true and rerun the installation.

[hhaensel]

... to me it seems that you have different access rights in your REPL and in the run($(Base.julia_cmd()))

... I'll be offline for a while

[zgornel]

@hhaensel , indeed, it looks like a permission problem as I cannot reproduce the problem on a second machine. So,

• libz.so is a symlink indeed, lilke libz.so.1 (both point to the same file)
• the ENV["BINARYPROVIDER_COPYDEREF"]=true or copying the lib in place of the symlink do not work
• wiping out the .julia directory solves the problem
• I do not see any way of tracing this issue back to something specific so far It may be that the problem is not related to BinaryProvider. Thanks for the assistance.

[hhaensel]

We came across a similar issue recently, when ImageMagick could not be rebuild without removing the binaries beforehand: ImageMagick, PR 159 and FileIO, Issue 229 And I realised in the discussion of PR 166 that - similar to @zgornel 's remark above - the first check with satisfied involves default isolate=false whereas write_deps_file involves default isolate=true. @staticfloat , maybe it would be good to establish some kind of version checking for binaries? Alternatively, one could think about a force option for build, so that installation is dependent on satisfied(...) || force = true?

[hhaensel]

@hhaensel , indeed, it looks like a permission problem as I cannot reproduce the problem on a second machine. So,

* `libz.so` is a symlink indeed, lilke `libz.so.1` (both point to the same file)

* the `ENV["BINARYPROVIDER_COPYDEREF"]=false` or copying the lib in place of the symlink do not work

* wiping out the `.julia` directory **solves the problem**

* I do not see any way of tracing this issue back to something specific so far
  It may be that the problem is not related to `BinaryProvider`. Thanks for the assistance.

It should have been ENV["BINARYPROVIDER_COPYDEREF"]=true ;-) But if you found that replacing the symlinks by hardcopies did not help the error has probably been elsewhere... It would be good to hear from @Crown421 and @BoundaryValueProblems whether their problems can be solved by removing the deps folder.

[BoundaryValueProblems]

@hhaensel, I just tried FreeType.jl by removing deps folder and building it via (v1.1) pkg> build FreeType. Nothing happened and deps was not created. On the other hand if I go back to BinaryProvider v0.5.4, every single packages is built without any problem. So, for a while, I have to stick with v0.5.4 until v0.5.5 is fixed.

[hhaensel]

Two questions:

• Did the error message change?
• Is removing (or temporarily renaming) the whole .julia folder an option as @zgornel wrote?

[staticfloat]

@staticfloat , maybe it would be good to establish some kind of version checking for binaries?

Indeed. We're getting there. ;)

[zgornel]

@hhaensel , indeed, it looks like a permission problem as I cannot reproduce the problem on a second machine. So,

* `libz.so` is a symlink indeed, lilke `libz.so.1` (both point to the same file)

* the `ENV["BINARYPROVIDER_COPYDEREF"]=false` or copying the lib in place of the symlink do not work

* wiping out the `.julia` directory **solves the problem**

* I do not see any way of tracing this issue back to something specific so far
  It may be that the problem is not related to `BinaryProvider`. Thanks for the assistance.

It should have been ENV["BINARYPROVIDER_COPYDEREF"]=true ;-) But if you found that replacing the symlinks by hardcopies did not help the error has probably been elsewhere...

My bad, the value was true (it did not use symlinks). Edited the comment for future ref.

[hhaensel]

@BoundaryValueProblems I was a bit fast in writing "delete the deps folder". I meant delete everything inside the deps folder except the build.jl. If this doesn't help or if you have already deleted the deps folder, please also delete the folder 9jDi1 inside CodecZlib and then run ]build -v JLD2 or ]build -v CodecZlib If there are any errors, please report them.

[hhaensel]

@staticfloat if isolate=true makes the difference, then it seems that the environments of the REPL and Base.julia_cmd() are different. And even stranger, according to @zgornel 's post above

dl_esc_path = "/home/zgornel/.julia/packages/CodecZlib/9jDi1/deps/usr/lib/libz.so"
 success(`$(Base.julia_cmd()) -e "import Libdl; Libdl.dlopen(\"$(dl_esc_path)\")"`)

returns true at the REPL, but not when run from the build process. I am afraid that anything concerning the package manager is above my abilities and I am probably out of the game here. If there is anything concerning the windows patch don't hesitate to involve me again ;-)

[hhaensel]

@staticfloat , maybe it would be good to establish some kind of version checking for binaries?

Indeed. We're getting there. ;)

O wow! That's indeed a big thing. Thanks for pointing to that.

[Crown421]

I deleted everything in the deps folder, and ran build -v CodecZlib and received the same error (on WSL)

Building CodecZlib → `~/.julia/packages/CodecZlib/9jDi1/deps/build.log`
[ Info: Downloading https://github.com/bicycle1885/ZlibBuilder/releases/download/v1.0.4/Zlib.v1.2.11.x86_64-linux-gnu.tar.gz to /home/steffen/.julia/packages/CodecZlib/9jDi1/deps/usr/downloads/Zlib.v1.2.11.x86_64-linux-gnu.tar.gz...
[14:07:24] ######################################################################## 100.0%
ERROR: LoadError: LibraryProduct(nothing, ["libz"], :libz, "Prefix(/home/steffen/.julia/packages/CodecZlib/9jDi1/deps/usr)") is not satisfied, cannot generate deps.jl!
Stacktrace:
 [1] error(::String) at ./error.jl:33

Same when deleting the entire 9jDi1 folder.

I can't test this right now on the linux server that I use, as I need it to run.

I should also mention that I put a completely fresh install of julia and all packages on the server the other day, and ran into the described issue creating my baseline environment. No updates or actual use had happened yet.

[hhaensel]

Very strange! Just did a fresh install of both julia-1.1.0 and julia-1.1.1 on WSL Ubuntu on an updated Windows 10 64bit and everything works fine. Perhaps rename your .julia for a while and retry ]add CodecZlib and build -v CodecZlib.

(v1.1) pkg> build -v CodecZlib
  Building CodecZlib → `~/.julia/packages/CodecZlib/9jDi1/deps/build.log`

(v1.1) pkg> st
    Status `~/.julia/environments/v1.1/Project.toml`
  [944b1d66] CodecZlib v0.5.2

(v1.1) pkg> st -m
    Status `~/.julia/environments/v1.1/Manifest.toml`
  [b99e7846] BinaryProvider v0.5.5
  [944b1d66] CodecZlib v0.5.2
  [3bb67fe8] TranscodingStreams v0.9.4
  [2a0f44e3] Base64
  [8ba89e20] Distributed
  [b77e0a4c] InteractiveUtils
  [8f399da3] Libdl
  [56ddb016] Logging
  [d6f4376e] Markdown
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [8dfed614] Test

[Crown421]

Once I am done with the current project I will give reinstalling julia on the server a try. Given that that the facts (deleting .julia helps, and I ran into the problem from a fresh install), some other package may create the conflict if installed first. Unfortunately I do not remember the order in which I installed the packages, but I will check the packages that I have and compare against your list.

[hhaensel]

That was a fresh installation. All packages listed in the manifest are installed by the ]add CodecZlib. Good luck!

[BoundaryValueProblems]

@hhaensel : I tried by moving my ~/.julia to ~/.julia.bak and installed various packages. But when it tried to install CodecZlib, it failed with the same error. I only tried this on my Windows 10 machine. By the way, I didn't reinstall julia v1.1.1; I just used the previously installed julia v1.1.1. Anyways, I will stick with [email protected] for a while.

[BoundaryValueProblems]

Today, I installed Julia v1.2.0, and hence, I also did fresh install of various packages. BinaryProvider v0.5.6 was installed automatically, and it didn't complain anything. So, perhaps, now is a good time to close this issue?

[hhaensel]

You are right, we can close this issue. The root cause is not yet solved, though. BinaryProvider v0.5.6 is just a rename of v0.5.4 in order to overcome the problems of the introduction of the isolate mode. There are some improvements in v0.5.5, such as symlinks on windows, which currently don't work. But this is pursued in #172, so feel free to close this :-)

[BoundaryValueProblems]

OK, then let's close this issue since #172 is still open. But somehow I myself cannot close it. Could someone close this? Thanks!