raft icon indicating copy to clipboard operation
raft copied to clipboard

WIP: Prevent path conflict in builds

Open AyodeAwe opened this issue 10 months ago • 1 comments

Contributes to https://github.com/rapidsai/build-planning/issues/54

This will make builds fail when there are path conflicts

AyodeAwe avatar Mar 25 '24 19:03 AyodeAwe

This is currently failing with the following conflicts.

(I've included just 1 example of each type below)

(CUDA 11.8 build log) (CUDA 12.2 build log)

  1. fmt and spdlog issues (conflicting packages: conda-forge/fmt, librmm, libraft-headers-only)

Described in https://github.com/rapidsai/rmm/pull/1508#issuecomment-2067300280.

  1. (CUDA 11.x only) CUDA toolkit packages packaged differently (conflicting packages: nvidia/cuda-{component}, conda-forge/cudatoolkit)
This transaction has incompatible packages due to a shared path.
  packages: nvidia/linux-64::libcublas-11.11.3.6-0, conda-forge/linux-64::cudatoolkit-11.8.0-h4ba93d1_13
  path: 'lib/libcublas.so.11'

And similar for all these:

lib/libcublas.so.11.11.3.6
lib/libcublasLt.so.11
lib/libcublasLt.so.11.11.3.6
lib/libnvblas.so.11
lib/libnvblas.so.11.11.3.6
lib/libcurand.so.10
lib/libcurand.so.10.3.0.86
lib/libcusolver.so.11
lib/libcusolver.so.11.4.1.48
lib/libcusolverMg.so.11
lib/libcusolverMg.so.11.4.1.48
lib/libcusparse.so.11
lib/libcusparse.so.11.7.5.86
lib/libcublas.so
lib/libcublasLt.so
lib/libnvblas.so
lib/libcurand.so
lib/libcusolver.so
lib/libcusolverMg.so
lib/libcusparse.so
  1. (CUDA 11.x only) lib-{component} and lib-{component}-dev from the nvidia channel
This transaction has incompatible packages due to a shared path.
  packages: nvidia/linux-aarch64::libcusparse-11.7.5.86-0, nvidia/linux-aarch64::libcusparse-dev-11.7.5.86-0, conda-forge/linux-aarch64::cudatoolkit-11.8.0-hdd48f4a_13
  path: 'lib/libcusparse.so.11'
  1. cub (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
  packages: rapidsai-nightly/linux-64::librmm-24.06.00a0-cuda11_240325_gfb23a13a_0, file:///tmp/conda-bld-output/linux-64::libraft-headers-only-24.06.00a24-cuda11_240325_g367052cd_24
  path: 'include/rapids/cub/agent/agent_adjacent_difference.cuh'
  1. libcudacxx (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.06.00a0-cuda11_240325_gfb23a13a_0, file:///tmp/conda-bld-output/linux-64::libraft-headers-only-24.06.00a24-cuda11_240325_g367052cd_24
  path: 'include/rapids/libcudacxx/cuda/memory_resource
  1. thrust (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
  packages: rapidsai-nightly/linux-64::librmm-24.06.00a0-cuda12_240325_gfb23a13a_0, file:///tmp/conda-bld-output/linux-64::libraft-headers-only-24.06.00a24-cuda12_240325_g367052cd_24
  path: 'include/rapids/thrust/adjacent_difference.h'
  1. rmm (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.06.00a0-cuda12_240325_gfb23a13a_0, file:///tmp/conda-bld-output/linux-64::libraft-headers-only-24.06.00a24-cuda12_240325_g367052cd_24
  path: 'include/rmm/aligned.hpp

jameslamb avatar Apr 23 '24 17:04 jameslamb

This work is paused, in favor of pursuing a better long-term solution in the future. Closing this PR for now.

Subscribe to https://github.com/rapidsai/build-planning/issues/54 and https://github.com/rapidsai/build-planning/issues/56 for updates.

jameslamb avatar Jul 18 '24 16:07 jameslamb