Use rapids-build-backend

[KyleFromNVIDIA]

Contributes to rapidsai/build-planning#31.

[bdice]

@KyleFromNVIDIA I'm going to merge the upstream into this PR to get a fresh CI run. This is the only open PR targeting 24.06 and it's a good candidate to look at. I want to investigate the latest CI failures, which @rjzamora indicated may be related to dask-expr.

I'll compare its failures to those on #5815.

[jameslamb]

Could you please use mentions in the PR description to link these PRs to the main issue they're contributing to? e.g. like this:

Contributes to https://github.com/rapidsai/build-planning/issues/31.

Think that'd help library teams to trace back to the context for these changes, and it makes it easier to go from the main issue to all of the changes across RAPIDS that it led to. I've found those types of links really useful in reconstructing the context for why something was done in the past.

[jameslamb]

Those 3 failing cudf-pandas-integration test jobs are optional and the failures are not a result of this PR's changes. I'm confident in that because I see the exact same failures on other unrelated PRs, like #5905.

  File "/opt/conda/envs/test/lib/python3.9/site-packages/cuml/preprocessing/LabelEncoder.py", line 121, in LabelEncoder
  Failed example:
      print(str_label)
  Exception raised:
      Traceback (most recent call last):
        File "/opt/conda/envs/test/lib/python3.9/doctest.py", line 1334, in __run
          exec(compile(example.source, filename, "single",
        File "<doctest LabelEncoder[17]>", line 1, in <module>
          print(str_label)
      NameError: name 'str_label' is not defined

(build link)

This PR looks like it has all the changes we want for the rapids-build-backend rollout, so I'm merging it.

[jameslamb]

/merge

[jameslamb]

Oh interesting, I see this now

Waiting on code owner review from rapidsai/cuml-cmake-codeowners and/or rapidsai/cuml-python-codeowners.

Even though I don't see those groups in the list of reviewers.

I guess the groups were dropped because @bdice and @vyasr are a part of them and left earlier reviews "on behalf of" those groups.

[jameslamb]

The devcontainer pip job is failing like this:

ERROR: Cannot install cuml-cu11 because these package versions have conflicting dependencies.

The conflict is caused by:
    raft-dask-cu11 24.8.0a20 depends on distributed-ucxx-cu11==0.39.*
    raft-dask-cu11 24.8.0a19 depends on distributed-ucxx-cu11==0.39.*

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

(build link)

I believe the root cause is bugs from https://github.com/rapidsai/raft/pull/2331, not caught in CI because of https://github.com/rapidsai/raft/issues/2348, which should be fixed by some combination of https://github.com/rapidsai/raft/pull/2349 and https://github.com/rapidsai/raft/pull/2347.

[jameslamb]

The only failing tests here are the optional pandas ones, and the failures look unrelated to the changes on this PR (https://github.com/rapidsai/cuml/pull/5804#issuecomment-2142439264). I also double-checked that there's no use of --find-links here (thinking about https://github.com/rapidsai/raft/issues/2348).

I'm confident this can be merged.

[jameslamb]

/merge