mamba icon indicating copy to clipboard operation
mamba copied to clipboard

How to set environment variables for a micromamba environment?

Open Yura52 opened this issue 2 years ago • 13 comments

Hi! Are there any plans to support the conda env config vars set/unset/list commands in micromamba? If not, what is the best way to emulate this functionality? I cannot find instructions on how to run custom scripts on (de)activation.

Yura52 avatar Aug 26 '22 12:08 Yura52

I'd be interested in that too. conda env config vars --help is a valid CLI, but micromamba env config vars list --help is not a thing, apparently.

vvzen avatar Aug 30 '22 12:08 vvzen

To help provide a bit more context, when you do conda env config vars set MY_ENV_VAR=1 conda will create (if needed) a $PREFIX/conda-meta/state file, and then populate it with a JSON object: {"env_vars": {"MY_ENV_VAR": "1"}.

On userland, the workflow looks like this:

$ conda env config vars set --prefix ./my-prefix MY_ENV_VAR=1
$ conda activate ./my-prefix
$ echo $MY_ENV_VAR
1

Relevant source code:

  • env config CLI: https://github.com/conda/conda/blob/main/conda_env/cli/main_config.py
  • env config set implementation: https://github.com/conda/conda/blob/main/conda_env/cli/main_vars.py#L118 and https://github.com/conda/conda/blob/91fe7f7476b1c0997bef5cf554c14258b1da7f2f/conda/core/prefix_data.py#L328

vvzen avatar Oct 04 '22 11:10 vvzen

@Yura52 @vvzen I have just implemented the necessary bits here: https://github.com/mamba-org/mamba/pull/1968

If you have time to help or review the code, that would be very appreciated! :)

wolfv avatar Oct 04 '22 11:10 wolfv

Nice!

does that also apply to run in Conda (and now mamba)?

jonashaag avatar Oct 04 '22 11:10 jonashaag

Thanks a lot @wolfv !! Happy to help.

I cloned your fork (my HEAD is at 512d501) and built micromamba locally, like this:

$ cmake .. -DBUILD_LIBMAMBA=ON -DBUILD_STATIC_DEPS=ON
$ make

The build process went fine, I now have:

$ /path/to/my/newly/built/micromamba/bin/micromamba --version
0.26.0

From a quick glance at your PR, it looks like you're just setting the env vars in conda-meta/state only at activation time, or am I mis-reading this ?

I can also see that micromamba env has got only list and export as subcommands. So I suppose that for now you added the implementation, but you have not yet exposed it to the micromamba CLI ?

Anyway, I gave it a try like this, since I thought that the env vars would be set at activation time:

# Write a sample JSON file:
$ echo '{"env_vars": {"MY_ENV_VAR": "1"}' >> ./my-prefix/conda-meta/state

# Activate the environment and check the result
$ export PATH="/path/to/my/newly/built/micromamba/bin:$PATH"
$ eval "$(micromamba shell hook --shell=bash)"
$ micromamba activate ./my-prefix/
$ env | grep -c MY_ENV_VAR
0

Am I doing this wrong? Thanks!

vvzen avatar Oct 04 '22 12:10 vvzen

@vvzen awesome! :) you need to set the proper $MAMBA_EXE or better, modify your ~/.bashrc (or whatever shell you are using) to point to the right executable that does the new things.

You can also look at the activation scripts by executing mamba shell activate -n myenv (that prints the scripts to the console).

wolfv avatar Oct 04 '22 12:10 wolfv

Sorry for the delay! My previous build success was a red herring - so I spent a good amount of time trying to get a static build of libmamba and then of micromamba (following first https://mamba.readthedocs.io/en/latest/developer_zone/build_locally.html#fully-static-library and then https://mamba.readthedocs.io/en/latest/developer_zone/build_locally.html#fully-statically-linked). With the added fun of being on an M1 machine and dealing with CMake that wanted to link against the ARM version of openssl@3, even though I needed a x86_64 build :)

This is what I have now:

$ export PATH="/path/to/my/newly/built/micromamba/bin:$PATH"
$ export MAMBA_EXE=/path/to/my/newly/built/micromamba/bin/micromamba
$ micromamba activate ./my-prefix
$ env | grep MY_ENV_VAR
MY_ENV_VAR=1

I can also see that:

$ micromamba shell activate ./mamba-prefix | grep MY_ENV_VAR
export MY_ENV_VAR='1'

So things seem to be working great! I'll play a bit more with this build to be check if everything runs smooth, thanks a lot!

vvzen avatar Oct 04 '22 15:10 vvzen

@vvzen

It should be

$ echo '{"env_vars": {"MY_ENV_VAR": "1"}}' >> ./my-prefix/conda-meta/state

you missed a "}". so the following error:

warning  libmamba Could not read JSON at ".../conda-meta/state": [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing object - unexpected end of input; expected '}'

heavenkiller2018 avatar Jul 17 '23 08:07 heavenkiller2018

What is the status of this issue?

phreed avatar Mar 11 '24 20:03 phreed

To summarize, is the recommended (and only) way to set environment variables per environment for micromamba to write json at $CONDA_PREFIX/conda-meta/state in the following format:

{
  "env_vars": {
    "MY_ENV_VAR": "1"
  }
}

where every key/value pair in env_vars correspond to an environment variable name/value? If I want to append to a pre-existing environment variable (like PATH), do I need to hack it really hard as follows?

{
  "env_vars": {
    "PATH": "/some/path:'$PATH'"
  }
}

Sinacam avatar Apr 09 '24 11:04 Sinacam

Should this instruction be added to the documentation, or should micromamaba be improved to pull this information from the environment.yaml when creating / updating a named environment?

phreed avatar Apr 12 '24 15:04 phreed

Since Conda supports setting custom environment variables directly in an environment's configuration.yaml file then I would suggest that Mamba / Micromamba support the same feature.

-- Edited * --

However since the configuration file is only used when the environment is created I appreciate this has limited use/appeal over (or in addition to) the ... env vars set/unset/list functionality implemented by @wolfv (which I missed before).

The consideration I have now is, if support for variables coming from the YAML files were to be implemented then we must also ensure that these (and any additions or modifications) are exported back when running ... env export.

That should probably be raised as a separate issue though(?!), and this issue (as stated in the OP) could be considered closed?
- Perhaps with some final comment here after the updates from this thread make it to a release?
- I assume they haven't yet because this functionality doesn't work for me!

* Apologies, still playing catch-up on a lot of things and I should have read through the whole issue more closely


micromamba info:

libmamba version : 1.5.8
micromamba version : 1.5.8
...
virtual packages : __win=0=0
__archspec=1=x86_64
channels : https://conda.anaconda.org/conda-forge/win-64
             https://conda.anaconda.org/conda-forge/noarch
...
platform : win-64

apitofme avatar Apr 16 '24 13:04 apitofme

Please forgive my misunderstanding...

Having looked at both the pull request #1968 and the resulting merged commit 53436fb it appears that support for setting environment variables from the CLI and from the YAML file have both been implemented, including exporting these back to the YAML file (as best I can tell).

I am not experienced with C++ however that is my understanding when looking at the activation.cpp file specifically, from that commit.

So if my interpretation is correct, and given that the commit occurred on "Oct 13, 2022", then I also assume that this should be part of every release since then, or at the very least by now? However this functionality still does not work for me:

  • The set / unset / list sub-commands are not present for the ... env command:
micromamba env -h
List environments
Usage: ... micromamba.exe env [OPTIONS] [SUBCOMMAND]
...
Subcommands:
  list                        List known environments
  create                      Create new environment (pre-commit.com compatibility alias for 'micromamba create')
  export                      Export environment
  remove                      Remove an environment
  • and attempting to use ... env vars set or (conda's style) ... env config vars set does not work:
micromamba env vars set MY_VAR=test
The following arguments were not expected: MY_VAR=test set vars
Run with --help for more information.
micromamba env config vars set MY_VAR=test
The following arguments were not expected: MY_VAR=test set vars config
Run with --help for more information.

Do I need to download and build micromamba from the latest source, or what am I missing? How can I get this functionality to work please?


... Given the tag micromamba-2.0.0beta0 from the aforementioned commit, does that mean these features haven't made it in to a release yet?
... Going through the releases pages specifically searching for #1968 yields no results, so I can only assume that indeed this has not yet made it in to a release.

apitofme avatar Apr 17 '24 12:04 apitofme