[BUG] Build script takes a very long time before it begins building targets.

[ZelboK]

Describe the bug It takes around 6-7 minutes before ./build.sh libcudf tests will begin building targets. There is a lot of preparation work that is done. This can be unproductive as the feedback loop of testing your code against the compiler is greatly inhibited by this delay.

Steps/Code to reproduce bug

1. ./build.sh clean (to remove old build artifacts, just incase they were being problematic
2. ./build.sh libcudf tests full build, 6 minutes until it starts building
3. ./build.sh libcudf tests run again to test how long it takes before targets actually start building.

Expected behavior I expected there to be variation in how long it would take after the first clean full build but there isn't. It is consistently that long.

Environment overview (please complete the following information)

• WSL2, 24 core CPU, 1gbit download connection, 3080 GPU(should be irrelevant for this tho)
• Parallel building is turned on(should be set to same amount of processors, 24, from what i see in the build script)
• Using conda with compilers GNU 11.4 & CUDA 12.0.

**Additional details Using conda and the build script is consistently slow for me. It is significantly faster if I do not rely on it. It takes around 30-40 seconds without conda. Around ~13x faster.

Tangential, but I also think that devcontainers being set up to use sccache locally since libcudf is a large project for those outside of NVIDIA could possibly be a very helpful way to improve compile times. CCCL monorepo devcontainers are currently set up to do exactly that. However this might require more effort with conda and pip in the picture here.

[bdice]

Can you tell what it’s doing during this 6-7 minute period? Check the running processes. For me, clean builds take less than a minute to start building. It takes a little bit to download the dependencies via CPM, but subsequent builds take only a few seconds. Do you have a reasonably fast internet connection for CPM downloads?

[ZelboK]

Wouldn't subsequent runs after the first one rule out downloading? With that being said the internet connection is at around 900 - 1100 mbps. Definitely not the problem.

Important to note again though that this only happens with conda environments. If I do not use conda environments it is normal. I will report back when I try out conda again, with what might be running in the BG.

[vyasr]

Do you see CMake starting to run, or is it taking a long time to even get to CMake?

[bdice]

Closing as stale. Feel free to reopen if more context can be provided.