cuml icon indicating copy to clipboard operation
cuml copied to clipboard

WIP: Prevent path conflict in builds

Open AyodeAwe opened this issue 11 months ago • 1 comments

Contributes to https://github.com/rapidsai/build-planning/issues/54

This will make builds fail when there are path conflicts

AyodeAwe avatar Mar 25 '24 19:03 AyodeAwe

This is currently failing with the following conflicts.

(I've included just 1 example of each type below)

(CUDA 11.8 build log) (CUDA 12.2 build log)

  1. fmt and spdlog issues (conflicting packages: conda-forge/fmt, librmm, libraft-headers-only)

Described in https://github.com/rapidsai/rmm/pull/1508#issuecomment-2067300280.

  1. (CUDA 11.x only) CUDA toolkit packages packaged differently (conflicting packages: nvidia/cuda-{component}, conda-forge/cudatoolkit)
This transaction has incompatible packages due to a shared path.
  packages: nvidia/linux-64::libcublas-11.11.3.6-0, conda-forge/linux-64::cudatoolkit-11.8.0-h4ba93d1_13
  path: 'lib/libcublas.so.11'

And similar for all these:

lib/libcublas.so.11.11.3.6
lib/libcublasLt.so.11
lib/libcublasLt.so.11.11.3.6
lib/libnvblas.so.11
lib/libnvblas.so.11.11.3.6
lib/libcufft.so.10
lib/libcufft.so.10.9.0.58
lib/libcufftw.so.10
lib/libcufftw.so.10.9.0.58
lib/libcurand.so.10
lib/libcurand.so.10.3.0.86
lib/libcusolver.so.11
lib/libcusolver.so.11.4.1.48
lib/libcusolverMg.so.11
lib/libcusolverMg.so.11.4.1.48
lib/libcusparse.so.11
lib/libcusparse.so.11.7.5.86
lib/libcublas.so
lib/libcublasLt.so
lib/libnvblas.so
lib/libcufft.so
lib/libcufftw.so
lib/libcurand.so
lib/libcusolver.so
lib/libcusolverMg.so
lib/libcusparse.so
  1. (CUDA 11.x only) lib-{component} and lib-{component}-dev from the nvidia channel
This transaction has incompatible packages due to a shared path.
  packages: nvidia/linux-64::libcusparse-11.7.5.86-0, nvidia/linux-64::libcusparse-dev-11.7.5.86-0, conda-forge/linux-64::cudatoolkit-11.8.0-h4ba93d1_13
 path: 'lib/libcusparse.so.11.7.5.86'
  1. cub *(conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
 path: 'include/rapids/cub/agent/agent_radix_sort_upsweep.cuh'
  1. libcudacxx (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
  path: 'include/rapids/libcudacxx/cuda/std/cassert'
  1. thrust (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
 path: 'include/rapids/thrust/addressof.h'
  1. rmm (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
  packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
  path: 'include/rmm/mr/device/arena_memory_resource.hpp'

jameslamb avatar Apr 23 '24 18:04 jameslamb

This work is paused, in favor of pursuing a better long-term solution in the future. Closing this PR for now.

Subscribe to https://github.com/rapidsai/build-planning/issues/54 and https://github.com/rapidsai/build-planning/issues/56 for updates.

jameslamb avatar Jul 18 '24 16:07 jameslamb