feat: conda/mamba create env from `env.yaml` file

[kemingy]

Signed-off-by: Keming [email protected]

• split "create user envd" and "chown envd"
• fix conda/mamba create new env from env YAML file

[muniu-bot[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kemingy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [kemingy]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[VoVAllen]

Also will conda install anything under current user folder?

[kemingy]

Do you need also chown on pip package path? Also if there's pip package in conda yaml file, is it installed under root user or envd user?

Pip package path should be under "/opt/conda/envs/envd".

Tested "install -e ." can work.

[kemingy]

Also will conda install anything under current user folder?

Does "current user folder" mean "/home/envd"? This one has also been chowned.

[VoVAllen]

I mean if the installation process is executed under root user, will it install anything to /root/XXXX?

[gaocegege]

Pip package path should be under "/opt/conda/envs/envd".

Could you please verify if pip cache works after the change? I am not sure if /home/envd/.pip/cache still work

[kemingy]

I mean if the installation process is executed under root user, will it install anything to /root/XXXX?

Not sure if some libraries will modify the /root/ dir. Do you have any examples? I tried jupyter, it looks good.

Pip package path should be under "/opt/conda/envs/envd".

Could you please verify if pip cache works after the change? I am not sure if /home/envd/.pip/cache still work

The cache seems doesn't work at all. I tried to use /root/.cache/pip, /opt/conda/pkgs, but it doesn't work. Let me try to figure it out.

[VoVAllen]

You can try install pip package such as numpy in conda. And check whether user can import numpy under user envd

[gaocegege]

And, if we use root in the building, and use envd in the runtime, there may be different cache locations between build time and runtime.

For example, pip cache in build time will be stored in /home/root/.pip, and it should be /home/envd/.pip in runtime.

I am not sure if it is possible to customise the cache location.

[gaocegege]

Please rebase.

[gaocegege]

https://github.com/tensorchord/envd/actions/runs/3170574497/jobs/5163236483#step:7:3192

There are some issues in CI.

[kemingy]

According to my observation, micromamba update -f env.yaml doesn't install anything from the YAML file. Instead, it only updates all the existing packages.

We'd better use conda update because it requires mounts, which means it cannot be cached.

[kemingy]

I removed most of the chown operations. Users can still access conda, use pip install.

• fix actions/toolkit#644

[kemingy]

https://open-vsx.org/api is too fragile. Sadly, GitHub actions don't support allow-failure https://github.com/actions/runner/issues/2347.

[gaocegege]

/lgtm

Thanks for your contribution! :tada: :+1: