[Core] extend conda runtime_env to allow for using mamba as per PR 33596

[cosnicolaou]

This is a starting point PR to find a means of allowing an alternative to conda to be used for creating environments. As per PR #33596 conda can be slow when building non-trivial environments. This PR allows for mamba to be used instead. It takes the form of extending the runtime_env's 'conda' dictionary to allow for an alternative executable to specified for creating environments:

runtime_env = {
        "conda": {"create_env_exe": "mamba", "dependencies": ["pip", {"pip": ["pip-install-test==0.5"]}]}
    }

The only functions that are altered are create_conda_env_if_needed to accept the name of exe to use rather than hardcoding "conda" and CondaPlugin to obtain the executable name from the runtime_env.

I'd like some guidance as to whether this is the best approach for addressing this issue before I go ahead and update the docs. Also, the unit test that I've added test_conda_using_mamba_create_task doesn't test that mamba is actually used, just that the package creation succeeds since I'm not sure how best to access more detailed logs from within tests.

Why are these changes needed?

See PR #33596.

Related issue number

Checks

• [ ] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• [x] I've run scripts/format.sh to lint the changes in this PR.
• [ ] I've included any doc changes needed for https://docs.ray.io/en/master/.
  • [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file.
• [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • [x] Unit tests
  • [ ] Release tests
  • [ ] This PR is not tested :(

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[cosnicolaou]

is this any interest in this?

[cosnicolaou]

My intention is to use mamba, or micromamba, instead of conda not in conjunction with it. With this change, the ray.init environment would specify which tool to use rather than having it be hardwired to conda as it is today.

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[Dessix]

This would be a rather nice improvement, considering how much faster Mamba runs; I would say this should be kept open for continuation.

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[Dessix]

This is still a worthwhile change; can any contributors take a look?

[jjyao]

Sorry for missing this PR. The team will review this soon!

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[Dessix]

@jjyao So, about that...