Remove need for a miniconda installation ?

[Luthaf]

trafficstars

It could be feasible to parse directly the metadata in the info folder to resolve the dependencies directly in Julia.

Pro:

• Remove the need for a full miniconda installation, and use less bandwidth;
• (Maybe) faster operations, as their is no need to shell out commands, and only the needed code will be used;

Con:

• Why rewrite the wheel?
• Writing a correct package manager is hard.

[dhoegh]

I use your package because it install the miniconda and therefore I do not think you should remove it. If you do could you make a new package? My purpose is to make my Julia work environment easy to install even when I depend on both matplotlib and IPython notebooks. Your package fulfill this purpose perfectly as my installation script is:

Pkg.add("Conda")
using Conda
# Installing the notebook interface
Conda.add("ipython-notebook")
# Installing the plotting library
Conda.add("matplotlib")
#################### restart terminal ####################
# installs notebook
Pkg.add("IJulia")
# Plotting library
Pkg.add("PyPlot")

This is a lot easier to tell people to do than, asking them to install the Miniconda manually and using cmd to manage packages and the space required is smaller than using the whole Anaconda distribution.

[Luthaf]

Removing miniconda will not prevent you from doing this. The same packages would be present, and if a Python distribution is needed by a package, it would be installed.

Conda.add("matplotlib")

Would still download and install Python, Numpy, matplotlib and any dependency.

The idea here is to remove this python distribution when it is not used (i.e. when managing binary dependencies for BinDeps). But as this issue would require a pretty big amount of work (writing a correct package manager is hard), and my time is limited, so Miniconda is not going away anytime soon.

[dhoegh]

I see, good to know. I am promoting your package as the easiest way of getting IJulia installed.

[dpsanders]

@dhoegh Maybe you know that now all you need to do is

Pkg.add("IJulia")

and that will automatically install Conda etc.

[dhoegh]

Yes, I pushed the use of Conda in IJulia, PyCall and PyPlot forward.

[dpsanders]

OK, thanks very much for doing that!

[jakirkham]

I would like to add one more con here. Namely, you are now maintaining your own version of conda, which has the potential to diverge with time. This becomes more important when you want to deal with packages that are conda maintained.

[cortner]

If I already have the full Anaconda distribution, then Conda.jl still tries to install miniconda:

julia> Conda.add("pyamg")
INFO: Downloading miniconda installer ...

Is this intended behaviour - why does it not simply use the existing installation? If I let this go ahead, will it mess up my existing anaconda installation?

Thanks.

[Luthaf]

Yes, this is the intended behaviour: Conda.jl is using it's own version of miniconda exactly to prevent it from messing up any existing anaconda installation (and to prevent the user from messing up the Conda.jl installation ^^).

I really should add this to the README, as this question comes very often =)

[cortner]

It would also be really useful if there could be some examples how to setup build.jl to choose install a Python dependency correctly, i.e., using the existing installation if it can be found and Conda.jl otherwise?

[hayd]

@Luthaf IJulia.jl seems to have a slightly different policy (it uses the global, potentially incomplete, ipython i.e. even if it's without jupyter) https://github.com/JuliaLang/IJulia.jl/issues/363#issuecomment-220792927

[Luthaf]

It would also be really useful if there could be some examples how to setup build.jl to choose install a Python dependency correctly, i.e., using the existing installation if it can be found and Conda.jl otherwise?

@stevengj just added functionality for that in PyCall. See https://github.com/JuliaLang/METADATA.jl/pull/5201 and links for the API.

@Luthaf IJulia.jl seems to have a slightly different policy (it uses the global, potentially incomplete, ipython i.e. even if it's without jupyter) JuliaLang/IJulia.jl#363 (comment)

As I wrote it, Conda.jl provide access to conda, not python. The fact that conda is written in Python is purely an implementation detail.

The main goal of Conda.jl is to provide a cross-platform access to native (C, C++ and Fortran) dependencies, not Python dependencies =). So it is completely up to users of Conda.jl to decide what to do with it.

The IJulia.jl installation error should have it's own issue. I think this is some kind of bad interaction between packages.

[cortner]

Yes - I looked at PyCall. the file is about 300 lines, and given enough time I could probably dissect it. I am more looking for a

if has_anaconda_installation()
   run(`conda install somepackage`)
else 
  Conda.add("somepackage")
end

[Luthaf]

I was more thinking about the pyimport_conda("<pip name>", "<conda name>") function from PyCall. It tries to import the python package from the first argument, and if that fails and PyCall is using Conda, then it installs the package. Else it displays a message to the user giving instructions about how to use PyCall with Conda.jl.

But you could try to implement has_anaconda_installation() like this:

try
    run(`conda install somepackage`)
catch
    Conda.add("somepackage")
end

Except it might not work all the time. If you are trying to get a Python dependency, I strongly recommend you use the PyCall.pyimport_conda function.

[cortner]

thanks. so I could go the other way round, use

try 
     pyimport_conda("<pip name>", "<conda name>")
catch
    run(`conda install some package`)
end

that way, if the package is already installed, I won't install it again?

[stevengj]

@cortner, the pyimport_conda function does the conda install for you if needed. That's the whole point.

Note that this is documented in the PyCall README ... no need to read the PyCall source code.

[stevengj]

(Note that the first argument of pyimport_conda is the name of the module you want to import; it has nothing to do with pip.)

[cortner]

@stevengj Thank you and sorry for missing that.

[Luthaf]

@cortner Could you please open a separated issue for your problem? This one is for discussing the implementation of the conda package manager to use. Thanks!

[manslogic]

Hi,I am new to Julia, Please help me to get around this problem, i am trying to install jupyter notebook, i am facing below problem.

log:

		julia> Conda.add("ipython-notebook")
		[ Info: Downloading miniconda installer ...
		ERROR: failed process: Process(`'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe' -Version 3 -NoProfile -Command "[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12; (New-Object System.Net.Webclient).DownloadFile('https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Windows-x86_64.exe', 'C:\Users\trinity\.julia\conda\3\installer.exe')"`, ProcessExited(3221225477)) [3221225477]
		Stacktrace:
		[1] error(::String, ::Base.Process, ::String, ::Int64, ::String) at .\error.jl:42
		[2] pipeline_error at .\process.jl:785 [inlined]
		[3] download(::String, ::String) at .\download.jl:20
		[4] _install_conda(::String, ::Bool) at C:\Users\trinity\.julia\packages\Conda\CpuvI\src\Conda.jl:160
		[5] _install_conda(::String) at C:\Users\trinity\.julia\packages\Conda\CpuvI\src\Conda.jl:152
		[6] runconda(::Cmd, ::String) at C:\Users\trinity\.julia\packages\Conda\CpuvI\src\Conda.jl:111
		[7] add at C:\Users\trinity\.julia\packages\Conda\CpuvI\src\Conda.jl:183 [inlined] (repeats 2 times)
		[8] top-level scope at none:0

[stevengj]

@manslogic, please don't post help requests to unrelated github issues. You are having a download problem; that might indicate a firewall or something on your machine. You could try asking for help on https://discourse.julialang.org/