CondaPkg.jl
CondaPkg.jl copied to clipboard
JuliaPy
Incorrect cuda-nvcc version
To install cuda-toolkit 11.7.1:
julia> CondaPkg.add("cuda-toolkit"; version="11.7.1", channel = "nvidia/label/cuda-11.7.1")
(test-condapkg) pkg> conda st
CondaPkg Status /home/ymtoo/Projects/tmp/test-condapkg/CondaPkg.toml
Environment
/home/ymtoo/Projects/tmp/test-condapkg/.CondaPkg/env
Packages
cuda-toolkit v11.7.1 (11.7.1, channel=nvidia/label/cuda-11.7.1)
shell> .CondaPkg/env/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Mon_Oct_24_19:12:58_PDT_2022
Cuda compilation tools, release 12.0, V12.0.76
Build cuda_12.0.r12.0/compiler.31968024_0
The installed packages:
$ conda list
# packages in environment at /home/ymtoo/Projects/tmp/test-condapkg/.CondaPkg/env:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
_sysroot_linux-64_curr_repodata_hack 3 h69a702a_13 conda-forge
alsa-lib 1.2.8 h166bdaf_0 conda-forge
binutils 2.40 hdd6e379_0 conda-forge
binutils_impl_linux-64 2.40 hf600244_0 conda-forge
binutils_linux-64 2.40 hbdbef99_1 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-compiler 1.6.0 hd590300_0 conda-forge
ca-certificates 2023.7.22 hbcca054_0 conda-forge
cuda-cccl 12.0.90 ha770c72_1 conda-forge
cuda-cccl-impl 2.0.1 ha770c72_0 conda-forge
cuda-cccl_linux-64 12.0.90 ha770c72_1 conda-forge
cuda-command-line-tools 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-compiler 12.0.0 h9bef630_0 conda-forge
cuda-cudart 12.0.107 h59595ed_6 conda-forge
cuda-cudart-dev 12.0.107 h59595ed_6 conda-forge
cuda-cudart-dev_linux-64 12.0.107 h59595ed_6 conda-forge
cuda-cudart-static 12.0.107 h59595ed_6 conda-forge
cuda-cudart-static_linux-64 12.0.107 h59595ed_6 conda-forge
cuda-cudart_linux-64 12.0.107 h59595ed_6 conda-forge
cuda-cuobjdump 12.0.76 h59595ed_0 conda-forge
cuda-cupti 12.0.90 h59595ed_1 conda-forge
cuda-cuxxfilt 12.0.76 h59595ed_0 conda-forge
cuda-documentation 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-driver-dev 12.0.107 h59595ed_6 conda-forge
cuda-driver-dev_linux-64 12.0.107 h59595ed_6 conda-forge
cuda-gdb 12.0.90 hd47b8d6_0 conda-forge
cuda-libraries 12.0.0 ha770c72_1 conda-forge
cuda-libraries-dev 12.0.0 ha770c72_1 conda-forge
cuda-memcheck 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-nsight 12.0.78 ha770c72_0 conda-forge
cuda-nsight-compute 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-nvcc 12.0.76 hba56722_10 conda-forge
cuda-nvcc-dev_linux-64 12.0.76 ha770c72_0 conda-forge
cuda-nvcc-impl 12.0.76 h59595ed_0 conda-forge
cuda-nvcc-tools 12.0.76 h59595ed_0 conda-forge
cuda-nvcc_linux-64 12.0.76 h8a487aa_10 conda-forge
cuda-nvdisasm 12.0.76 h59595ed_0 conda-forge
cuda-nvml-dev 12.0.76 hcb278e6_0 conda-forge
cuda-nvprof 12.0.90 h59595ed_0 conda-forge
cuda-nvprune 12.0.76 h59595ed_0 conda-forge
cuda-nvrtc 12.0.76 h59595ed_1 conda-forge
cuda-nvrtc-dev 12.0.76 h59595ed_1 conda-forge
cuda-nvtx 12.0.76 hcb278e6_0 conda-forge
cuda-nvvp 12.0.90 h59595ed_0 conda-forge
cuda-opencl 12.0.76 h59595ed_0 conda-forge
cuda-opencl-dev 12.0.76 ha770c72_0 conda-forge
cuda-profiler-api 12.0.76 ha770c72_0 conda-forge
cuda-sanitizer-api 12.0.90 h59595ed_0 conda-forge
cuda-toolkit 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-tools 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-version 12.0 hffde075_2 conda-forge
cuda-visual-tools 11.7.1 0 nvidia/label/cuda-11.7.1
cxx-compiler 1.6.0 h00ab1b0_0 conda-forge
dbus 1.13.6 h5008d03_3 conda-forge
expat 2.5.0 hcb278e6_1 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
freetype 2.12.1 hca18f0e_1 conda-forge
gcc 12.3.0 h8d2909c_1 conda-forge
gcc_impl_linux-64 12.3.0 he2b93b0_0 conda-forge
gcc_linux-64 12.3.0 h76fc315_1 conda-forge
gds-tools 1.5.0.59 hcb278e6_0 conda-forge
gettext 0.21.1 h27087fc_0 conda-forge
gmp 6.2.1 h58526e2_0 conda-forge
gxx 12.3.0 h8d2909c_1 conda-forge
gxx_impl_linux-64 12.3.0 he2b93b0_0 conda-forge
gxx_linux-64 12.3.0 h8a814eb_1 conda-forge
icu 72.1 hcb278e6_0 conda-forge
kernel-headers_linux-64 3.10.0 h4a8ded7_13 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
krb5 1.19.3 h08a2579_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
libcublas 12.0.1.189 hcb278e6_2 conda-forge
libcublas-dev 12.0.1.189 hcb278e6_2 conda-forge
libcufft 11.0.0.21 hcb278e6_1 conda-forge
libcufft-dev 11.0.0.21 hcb278e6_1 conda-forge
libcufile 1.5.0.59 hcb278e6_0 conda-forge
libcufile-dev 1.5.0.59 hcb278e6_0 conda-forge
libcurand 10.3.1.50 hcb278e6_0 conda-forge
libcurand-dev 10.3.1.50 hcb278e6_0 conda-forge
libcusolver 11.4.2.57 hcb278e6_1 conda-forge
libcusolver-dev 11.4.2.57 hcb278e6_1 conda-forge
libcusparse 12.0.0.76 hcb278e6_1 conda-forge
libcusparse-dev 12.0.0.76 hcb278e6_1 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libexpat 2.5.0 hcb278e6_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-devel_linux-64 12.3.0 h8bca6fd_0 conda-forge
libgcc-ng 13.1.0 he5830b7_0 conda-forge
libglib 2.76.4 hebfc3b9_0 conda-forge
libgomp 13.1.0 he5830b7_0 conda-forge
libiconv 1.17 h166bdaf_0 conda-forge
libnpp 12.0.0.30 h59595ed_0 conda-forge
libnpp-dev 12.0.0.30 h59595ed_0 conda-forge
libnuma 2.0.16 h0b41bf4_1 conda-forge
libnvjitlink 12.0.76 hcb278e6_1 conda-forge
libnvjitlink-dev 12.0.76 hcb278e6_1 conda-forge
libnvjpeg 12.0.0.28 hcb278e6_0 conda-forge
libnvjpeg-dev 12.0.0.28 ha770c72_0 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libsanitizer 12.3.0 h0f45ef3_0 conda-forge
libsqlite 3.42.0 h2797004_0 conda-forge
libstdcxx-devel_linux-64 12.3.0 h8bca6fd_0 conda-forge
libstdcxx-ng 13.1.0 hfd8a6a1_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libxcb 1.15 h0b41bf4_0 conda-forge
libxkbcommon 1.5.0 h5d7e998_3 conda-forge
libxkbfile 1.1.0 h166bdaf_1 conda-forge
libxml2 2.11.4 h0d562d8_0 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
ncurses 6.4 hcb278e6_0 conda-forge
nsight-compute 2022.4.0.15 h1166182_0 conda-forge
nspr 4.35 h27087fc_0 conda-forge
nss 3.89 he45b914_0 conda-forge
ocl-icd 2.3.1 h7f98852_0 conda-forge
openssl 3.1.2 hd590300_0 conda-forge
pcre2 10.40 hc3806b6_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
sysroot_linux-64 2.17 h4a8ded7_13 conda-forge
wayland 1.22.0 h8c25dac_1 conda-forge
xcb-util 0.4.0 hd590300_1 conda-forge
xcb-util-image 0.4.0 h8ee46fc_1 conda-forge
xcb-util-keysyms 0.4.0 h8ee46fc_1 conda-forge
xcb-util-renderutil 0.3.9 hd590300_1 conda-forge
xcb-util-wm 0.4.1 h8ee46fc_1 conda-forge
xkeyboard-config 2.39 hd590300_0 conda-forge
xorg-compositeproto 0.4.2 h7f98852_1001 conda-forge
xorg-damageproto 1.2.1 h7f98852_1002 conda-forge
xorg-fixesproto 5.0 h7f98852_1002 conda-forge
xorg-inputproto 2.3.2 h7f98852_1002 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.1.1 hd590300_0 conda-forge
xorg-libsm 1.2.4 h7391055_0 conda-forge
xorg-libx11 1.8.6 h8ee46fc_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxcomposite 0.4.6 h0b41bf4_1 conda-forge
xorg-libxdamage 1.1.5 h7f98852_1 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge
xorg-libxi 1.7.10 h7f98852_0 conda-forge
xorg-libxrandr 1.5.2 h7f98852_1 conda-forge
xorg-libxrender 0.9.11 hd590300_0 conda-forge
xorg-libxtst 1.2.3 h7f98852_1002 conda-forge
xorg-randrproto 1.5.0 h7f98852_1001 conda-forge
xorg-recordproto 1.14.2 h7f98852_1002 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-util-macros 1.19.3 h7f98852_0 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
By adding channels = ["anaconda"] to the CondaPkg.toml, the correct cuda-nvcc version is installed.
julia> conda up
(test-condapkg) pkg> conda st
CondaPkg Status /home/ymtoo/Projects/tmp/test-condapkg/CondaPkg.toml
Environment
/home/ymtoo/Projects/tmp/test-condapkg/.CondaPkg/env
Packages
cuda-toolkit v11.7.1 (11.7.1, channel=nvidia/label/cuda-11.7.1)
Channels
anaconda
shell> .CondaPkg/env/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Jun__8_16:49:14_PDT_2022
Cuda compilation tools, release 11.7, V11.7.99
Build cuda_11.7.r11.7/compiler.31442593_0
The installed packages:
$ conda list
# packages in environment at /home/ymtoo/Projects/tmp/test-condapkg/.CondaPkg/env:
#
# Name Version Build Channel
cuda-cccl 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-command-line-tools 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-compiler 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-cudart 11.7.99 0 nvidia/label/cuda-11.7.1
cuda-cudart-dev 11.7.99 0 nvidia/label/cuda-11.7.1
cuda-cuobjdump 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-cupti 11.7.101 0 nvidia/label/cuda-11.7.1
cuda-cuxxfilt 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-documentation 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-driver-dev 11.7.99 0 nvidia/label/cuda-11.7.1
cuda-gdb 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-libraries 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-libraries-dev 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-memcheck 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-nsight 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-nsight-compute 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-nvcc 11.7.99 0 nvidia/label/cuda-11.7.1
cuda-nvdisasm 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-nvml-dev 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-nvprof 11.7.101 0 nvidia/label/cuda-11.7.1
cuda-nvprune 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-nvrtc 11.7.99 0 nvidia/label/cuda-11.7.1
cuda-nvrtc-dev 11.7.99 0 nvidia/label/cuda-11.7.1
cuda-nvtx 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-nvvp 11.7.101 0 nvidia/label/cuda-11.7.1
cuda-sanitizer-api 11.7.91 0 nvidia/label/cuda-11.7.1
cuda-toolkit 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-tools 11.7.1 0 nvidia/label/cuda-11.7.1
cuda-visual-tools 11.7.1 0 nvidia/label/cuda-11.7.1
gds-tools 1.3.1.18 0 nvidia/label/cuda-11.7.1
libcublas 11.10.3.66 0 nvidia/label/cuda-11.7.1
libcublas-dev 11.10.3.66 0 nvidia/label/cuda-11.7.1
libcufft 10.7.2.91 0 nvidia/label/cuda-11.7.1
libcufft-dev 10.7.2.91 0 nvidia/label/cuda-11.7.1
libcufile 1.3.1.18 0 nvidia/label/cuda-11.7.1
libcufile-dev 1.3.1.18 0 nvidia/label/cuda-11.7.1
libcurand 10.2.10.91 0 nvidia/label/cuda-11.7.1
libcurand-dev 10.2.10.91 0 nvidia/label/cuda-11.7.1
libcusolver 11.4.0.1 0 nvidia/label/cuda-11.7.1
libcusolver-dev 11.4.0.1 0 nvidia/label/cuda-11.7.1
libcusparse 11.7.4.91 0 nvidia/label/cuda-11.7.1
libcusparse-dev 11.7.4.91 0 nvidia/label/cuda-11.7.1
libnpp 11.7.4.75 0 nvidia/label/cuda-11.7.1
libnpp-dev 11.7.4.75 0 nvidia/label/cuda-11.7.1
libnvjpeg 11.8.0.2 0 nvidia/label/cuda-11.7.1
libnvjpeg-dev 11.8.0.2 0 nvidia/label/cuda-11.7.1
nsight-compute 2022.2.1.3 0 nvidia/label/cuda-11.7.1
Julia and the package version:
julia> versioninfo()
Julia Version 1.9.2
Commit e4ee485e909 (2023-07-05 09:39 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 8 × Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
Threads: 1 on 8 virtual cores
(test-condapkg) pkg> st
Status `~/Projects/tmp/test-condapkg/Project.toml`
[992eb4ea] CondaPkg v0.2.18
(test-condapkg) pkg> st --manifest
Status `~/Projects/tmp/test-condapkg/Manifest.toml`
[992eb4ea] CondaPkg v0.2.18
[692b3bcd] JLLWrappers v1.4.1
[0f8b85d8] JSON3 v1.13.2
[0b3b1443] MicroMamba v0.1.14
[69de0a69] Parsers v2.7.2
[fa939f87] Pidfile v1.3.0
[aea7be01] PrecompileTools v1.1.2
[21216c6a] Preferences v1.4.0
[6c6a2e73] Scratch v1.2.0
[856f2bd8] StructTypes v1.10.0
[f8abcde7] micromamba_jll v1.4.9+0
[0dad84c5] ArgTools v1.1.1
[56f22d72] Artifacts
[2a0f44e3] Base64
[ade2ca70] Dates
[f43a241f] Downloads v1.6.0
[7b1f6079] FileWatching
[b77e0a4c] InteractiveUtils
[4af54fe1] LazyArtifacts
[b27032c2] LibCURL v0.6.3
[76f85450] LibGit2
[8f399da3] Libdl
[56ddb016] Logging
[d6f4376e] Markdown
[a63ad114] Mmap
[ca575930] NetworkOptions v1.2.0
[44cfe95a] Pkg v1.9.2
[de0858da] Printf
[3fa0cd96] REPL
[9a3f8284] Random
[ea8e919c] SHA v0.7.0
[9e88b42a] Serialization
[6462fe0b] Sockets
[fa267f1f] TOML v1.0.3
[a4e569a6] Tar v1.10.0
[8dfed614] Test
[cf7118a7] UUIDs
[4ec0a83e] Unicode
[deac9b47] LibCURL_jll v7.84.0+0
[29816b5a] LibSSH2_jll v1.10.2+0
[c8ffd9c3] MbedTLS_jll v2.28.2+0
[14a3606d] MozillaCACerts_jll v2022.10.11
[83775a58] Zlib_jll v1.2.13+0
[8e850ede] nghttp2_jll v1.48.0+0
[3f19e933] p7zip_jll v17.4.0+0
What about channels = ["nvidia"] or channels = ["nvidia/label/cuda-11.7.1"]?
The underlying issue is that CondaPkg passes -c conda-forge if you don't specify any channels, and so it's finding a bunch of compatible packages from the conda-forge channel with higher version numbers, and therefore installing those.
Currently CondaPkg specifies --no-channel-priority meaning that all channels are given equal priority. A future CondaPkg could specify a priority (e.g. put conda-forge at the bottom) which could mean that nvidia has higher priority than conda-forge.
channels = ["nvidia/label/cuda-11.7.1"] yields the correct cuda-nvcc version. For the following CondaPkg.toml
channels = ["nvidia/label/cuda-11.7.1", "conda-forge"]
[deps.cuda-toolkit]
channel = "nvidia/label/cuda-11.7.1"
version = "11.7.1"
conda up installs the wrong cuda-nvcc. This prevents the installation of other packages from conda-forge.
To clarify, CondaPkg installs the correct cuda-toolkit version from "nvidia/label/cuda-11.7.1" but it installs the dependencies, e.g., cuda-nvcc, with higher version numbers from "conda-forge". Am I right?