RasterIO.jl icon indicating copy to clipboard operation
RasterIO.jl copied to clipboard

Put the name of the GDAL shared lib into a variable so it can accommo…

Open joa-quim opened this issue 9 years ago • 10 comments

…date the different names it has on Windows and *nix

joa-quim avatar Nov 22 '15 15:11 joa-quim

Can you make the changes in https://github.com/wkearn/RasterIO.jl/blob/master/deps/build.jl instead?

yeesian avatar Nov 22 '15 17:11 yeesian

Sorry, I don't understand. I don't want to change build.jl (and to do what?) and besides a side effect of my patch is to, as an alternative, avoid the need to download dependencies via conda (which have failed for me both here and the NetCDF package).

joa-quim avatar Nov 22 '15 17:11 joa-quim

And sorry again, I didn't want to close this issue.

joa-quim avatar Nov 22 '15 17:11 joa-quim

Sorry, I don't understand. I don't want to change build.jl (and to do what?) and besides a side effect of my patch is to, as an alternative, avoid the need to download dependencies via conda (which have failed for me both here and the NetCDF package).

You might want to spend some time looking through the README for BinDeps.

yeesian avatar Nov 22 '15 17:11 yeesian

Just for clarification, libgdal actually is a variable which is set when the file deps/deps.jl is executed. This file is automatically generated upon a successful Pkg.build("RasterIO") and will include the exact name and path of the lib, so should work on Windows, too.

If you have a custom gdal.dll or netcdf.dll, make sure they can be found by julia (test with find_library) and run Pkg.build("RasterIO"). This should create a deps.jl file containing the path to your shared lib file. Please let me know if it doesn't. After that using RasterIO should just work.

meggart avatar Nov 23 '15 11:11 meggart

Thanks, I did not know these things but I'm still a bit confused. First, when I first installed RasterIO (and NetCDF btw) there was some build error and deps.jl file was not generated. Now, I repeated the Pkg.build("RasterIO") and all went well.

I can see now that I can edit deps.jl and change the path to use the lib that I want (the one installed by conda is minimalist to maximum, no extra drivers). But is there anyway to tell Pkg.build() to NOT download any GDAL and use the a certain one if it is found in the system?

See, this works for me

julia> Libdl.find_library(["gdal_w64"])
"gdal_w64"

but it still made conda download the other dll and put it in deps.jl

joa-quim avatar Nov 23 '15 16:11 joa-quim

Ok, I guess that BinDeps does not recognize the name gdal_w64. Can you try to add it in build.jl (line 5) under "aliases" and try if the build script still tries to install the Conda dependency? We should definitely add common alternative Windows library names there.

What is the exact name of your netcdf.dll?

meggart avatar Nov 23 '15 16:11 meggart

Right, I was dumb in my previous attempt. Meanwhile I advanced a bit more. Like you say I gdal_w64 to aliases and, if I move away the meanwhile downloaded, gdal111.dll out of the way, than the rebuild put the right dll in deps.jl. I guess this means conda would not had downloaded in first place. But it's now too late for me to test in this system.

Regarding the names, http://www.gisinternals.com/ has daily builds of GDAL and they use the official naming (by that I mean they use what's in GDAL's nmake.opt) which is to add a suffix with version numbering. Its me who also builds GDAL and netCDF for GMT (the cmake scripts now used by ncar, originated in GMT) that use a different naming that saved me lots of troubles with DLLs hell. That naming schema consists in adding the _w64 or _w32 suffix to dlls. So, when a Windows user installs GMT he/she will have an updated GDAL and netCDF dependencies using that convention. But I repeat, that is what I do for GMT.

joa-quim avatar Nov 23 '15 17:11 joa-quim

OK, back from another system and news. Installed a fresh julia and

julia> Pkg.init()
INFO: Initializing package repository c:\j\.julia\v0.4
INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl

julia> Pkg.clone("git://github.com/wkearn/RasterIO.jl")
INFO: Cloning RasterIO from git://github.com/wkearn/RasterIO.jl
INFO: Computing changes...
INFO: Updating cache of BinDeps...
INFO: Updating cache of Compat...
INFO: Cloning cache of Conda from git://github.com/Luthaf/Conda.jl.git
INFO: Cloning cache of JSON from git://github.com/JuliaLang/JSON.jl.git
INFO: Updating cache of SHA...
INFO: Updating cache of URIParser...
INFO: Installing BinDeps v0.3.19
INFO: Installing Compat v0.7.8
INFO: Installing Conda v0.1.8
INFO: Installing JSON v0.5.0
INFO: Installing SHA v0.1.2
INFO: Installing URIParser v0.1.1

next, edited build.jl and changed it to

sizeof(Int) == 8 ? other_gdal = "gdal_w64" : other_gdal = "gdal_w32"
libgdal = library_dependency("libgdal",aliases=["gdal","gdal111",other_gdal])

Some troubles of unknown origin

julia> Pkg.build("RasterIO")
INFO: Building RasterIO
INFO: Recompiling stale cache file c:\j\.julia\lib\v0.4\Conda.ji for module Cond
a.
WARNING: Module BinDeps uuid did not match cache file
==============================[ ERROR: RasterIO ]===============================


LoadError: __precompile__(true) but require failed to create a precompiled cache
 file
while loading c:\j\.julia\v0.4\RasterIO\deps\build.jl, in expression starting on
 line 8

================================================================================


================================[ BUILD ERRORS ]================================


WARNING: RasterIO had build errors.

insist

julia> Pkg.build("RasterIO")
INFO: Building RasterIO
INFO: Downloading miniconda installer ...
INFO: Installing miniconda ...

and voilá, in deps.jl

# Load dependencies
@checked_lib libgdal "C:\\progs_cygw\\GMTdev\\gmt5\\trunk\\WIN64\\bin\\gdal_w64.DLL"

and no Conda download.

joa-quim avatar Nov 23 '15 17:11 joa-quim

Indeed I also found that Pkg.build needs to be called twice before it works. It seems to be related to https://github.com/JuliaLang/julia/pull/13506. However this is merged on master, and both 0.4 and nightly Travis fail without running Pkg.build twice, as tested in https://github.com/visr/GDAL.jl/pull/9.

visr avatar Nov 23 '15 17:11 visr