Unable to cache available package list in jupyter when mamba is installed

[tlvu]

trafficstars

Description

Since we added mamba into our conda env, we are unable to update the cache of available package list. The extension used to work properly before. I removed mamba and the extension works again !

{
   "command" : "/opt/conda/bin/mamba repoquery search * --json",
   "conda_info" : {
      "GID" : 1000,
      "UID" : 1000,
      "active_prefix" : null,
      "active_prefix_name" : null,
      "av_data_dir" : "/opt/conda/etc/conda",
      "av_metadata_url_base" : null,
      "channels" : [
         "https://repo.anaconda.com/pkgs/main/linux-64",
         "https://repo.anaconda.com/pkgs/main/noarch",
         "https://repo.anaconda.com/pkgs/r/linux-64",
         "https://repo.anaconda.com/pkgs/r/noarch"
      ],
      "conda_build_version" : "not installed",
      "conda_env_version" : "4.11.0",
      "conda_location" : "/opt/conda/lib/python3.9/site-packages/conda",
      "conda_prefix" : "/opt/conda",
      "conda_private" : false,
      "conda_shlvl" : -1,
      "conda_version" : "4.11.0",
      "config_files" : [],
      "default_prefix" : "/opt/conda",
      "env_vars" : {
         "CIO_TEST" : "<not set>",
         "CONDA_ROOT" : "/opt/conda",
         "CURL_CA_BUNDLE" : "<not set>",
         "PATH" : "/opt/conda/envs/birdy/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
         "REQUESTS_CA_BUNDLE" : "<not set>",
         "SSL_CERT_FILE" : "<not set>"
      },
      "envs" : [],
      "envs_dirs" : [
         "/opt/conda/envs",
         "/home/jenkins/.conda/envs"
      ],
      "netrc_file" : null,
      "offline" : false,
      "pkgs_dirs" : [
         "/opt/conda/pkgs",
         "/home/jenkins/.conda/pkgs"
      ],
      "platform" : "linux-64",
      "python_version" : "3.9.5.final.0",
      "rc_path" : "/home/jenkins/.condarc",
      "requests_version" : "2.26.0",
      "root_prefix" : "/opt/conda",
      "root_writable" : true,
      "site_dirs" : [],
      "sys.executable" : "/opt/conda/bin/python",
      "sys.prefix" : "/opt/conda",
      "sys.version" : "3.9.5 (default, Jun  4 2021, 12:28:51) \n[GCC 7.5.0]",
      "sys_rc_path" : "/opt/conda/.condarc",
      "user_agent" : "conda/4.11.0 requests/2.26.0 CPython/3.9.5 Linux/4.15.0-163-generic debian/10 glibc/2.28",
      "user_rc_path" : "/home/jenkins/.condarc",
      "virtual_pkgs" : [
         [
            "__linux",
            "4.15.0",
            "0"
         ],
         [
            "__glibc",
            "2.28",
            "0"
         ],
         [
            "__unix",
            "0",
            "0"
         ],
         [
            "__archspec",
            "1",
            "x86_64"
         ]
      ]
   },
   "error" : "RuntimeError('LockFile error. Aborting.')",
   "exception_name" : "RuntimeError",
   "exception_type" : "<class 'RuntimeError'>",
   "traceback" : "Traceback (most recent call last):\n  File \"/opt/conda/lib/python3.9/site-packages/conda/exceptions.py\", line 1080, in __call__\n    return func(*args, **kwargs)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 917, in exception_converter\n    raise e\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 911, in exception_converter\n    exit_code = _wrapped_main(*args, **kwargs)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 869, in _wrapped_main\n    result = do_call(args, p)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 744, in do_call\n    exit_code = repoquery(args, parser)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 686, in repoquery\n    pool = repoquery_api.create_pool(channels, platform, use_installed)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/repoquery.py\", line 47, in create_pool\n    load_channels(\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/utils.py\", line 122, in load_channels\n    index = get_index(\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/utils.py\", line 103, in get_index\n    is_downloaded = dlist.download(True)\nRuntimeError: LockFile error. Aborting.\n"
}

Reproduce

1. Go to '...'
2. Click on '...'
3. Scroll down to '...'
4. See error '...'

Expected behavior

Context

• Python package version: Python 3.7.12
• Extension version: mamba_gator 5.1.2 pyhd8ed1ab_0 conda-forge
• General information:

$ conda run -n birdy jupyter labextension list
JupyterLab v3.2.5
/opt/conda/envs/birdy/share/jupyter/labextensions
        jupyter-leaflet v0.15.0 enabled OK
        jupyterlab-plotly v5.5.0 enabled OK
        jupyterlab-topbar-extension v0.6.1 enabled OK (python, jupyterlab-topbar)
        jupyterlab-jupytext v1.3.8 enabled OK (python, jupytext)
        jupyterlab-logout v0.5.0 enabled OK (python, jupyterlab-logout)
        jupyterlab-system-monitor v0.8.0 enabled OK (python, jupyterlab-system-monitor)
        nbdime-jupyterlab v2.1.1 enabled OK
        @mamba-org/gator-lab v3.0.2 enabled OK (python, mamba_gator)
        @jupyterlab/git v0.34.0 enabled OK (python, jupyterlab-git)
        @bokeh/jupyter_bokeh v3.0.4 enabled OK (python, jupyter_bokeh)
        @voila-dashboards/jupyterlab-preview v2.1.0 enabled OK (python, voila)
        @jupyter-server/resource-usage v0.5.0 enabled OK (python, jupyter-resource-usage)
        @pyviz/jupyterlab_pyviz v2.1.0 enabled OK (python, pyviz_comms)
        @jupyter-widgets/jupyterlab-manager v3.0.1 enabled OK (python, jupyterlab_widgets)
        @hadim/jupyter-archive v3.2.1 enabled OK (python, jupyter-archive)

Other labextensions (built into JupyterLab)
   app dir: /opt/conda/envs/birdy/share/jupyter/lab
        jupyterlab-dash v0.4.0 enabled OK


$ conda info

     active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /root/.condarc
 populated config files : /root/.condarc
          conda version : 4.11.0
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __linux=4.15.0=0
                          __glibc=2.28=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.11.0 requests/2.26.0 CPython/3.9.5 Linux/4.15.0-163-generic debian/10 glibc/2.28
                UID:GID : 0:0
             netrc file : None
           offline mode : False

Command Line Output

Paste the output from your command line running `jupyter lab` here, use `--debug` if possible.

Browser Output

Paste the output from your browser Javascript console here.

[tlvu]

Mamba was installed in the base conda env while the Jupyter is installed in another conda env. Should mamba be installed in the same conda env as the Jupyter?

[fcollonval]

Sorry for the late answer.

Mamba was installed in the base conda env while the Jupyter is installed in another conda env. Should mamba be installed in the same conda env as the Jupyter?

No it should not.

"error" : "RuntimeError('LockFile error. Aborting.')",

Could you try running the following command in a terminal on the machine?

/opt/conda/bin/mamba repoquery search * --json

[tlvu]

Could you try running the following command in a terminal on the machine?

/opt/conda/bin/mamba repoquery search * --json

@fcollonval Sorry for the late reply, from which dir I should run that command? From / I get this error:

usage: mamba [-h] [-V] command ...
mamba: error: unrecognized arguments: boot dev environment.yml etc home lib lib64 media mnt notebook_dir opt proc root run sbin srv sys tmp usr var

[tlvu]

Might be related, mamba install pyston_lite_autoload -c pyston -c conda-forge I get this error

    Traceback (most recent call last):
      File "/opt/conda/lib/python3.9/site-packages/conda/exceptions.py", line 1114, in __call__
        return func(*args, **kwargs)
      File "/opt/conda/lib/python3.9/site-packages/mamba/mamba.py", line 935, in exception_converter
        raise e
      File "/opt/conda/lib/python3.9/site-packages/mamba/mamba.py", line 929, in exception_converter
        exit_code = _wrapped_main(*args, **kwargs)
      File "/opt/conda/lib/python3.9/site-packages/mamba/mamba.py", line 887, in _wrapped_main
        result = do_call(args, p)
      File "/opt/conda/lib/python3.9/site-packages/mamba/mamba.py", line 750, in do_call
        exit_code = install(args, parser, "install")
      File "/opt/conda/lib/python3.9/site-packages/mamba/mamba.py", line 497, in install
        index = load_channels(pool, channels, repos)
      File "/opt/conda/lib/python3.9/site-packages/mamba/utils.py", line 129, in load_channels
        index = get_index(
      File "/opt/conda/lib/python3.9/site-packages/mamba/utils.py", line 110, in get_index
        is_downloaded = dlist.download(api.MAMBA_DOWNLOAD_FAILFAST)
    RuntimeError: Operation not permitted: '/opt/conda/pkgs/cache/47929eba.json'

But the same using conda works: conda install pyston_lite_autoload -c pyston -c conda-forge

conda-4.13.0 mamba-0.24.0

[tlvu]

@fcollonval I think I got what you wanted

$ /opt/conda/bin/mamba repoquery search birdy --json

{
  "command": "/opt/conda/bin/mamba repoquery search birdy --json",
  "conda_info": {
    "GID": 1000,
    "UID": 1000,
    "active_prefix": "/opt/conda/envs/birdy",
    "active_prefix_name": "birdy",
    "av_data_dir": "/opt/conda/etc/conda",
    "av_metadata_url_base": null,
    "channels": [
      "https://repo.anaconda.com/pkgs/main/linux-64",
      "https://repo.anaconda.com/pkgs/main/noarch",
      "https://repo.anaconda.com/pkgs/r/linux-64",
      "https://repo.anaconda.com/pkgs/r/noarch"
    ],
    "conda_build_version": "not installed",
    "conda_env_version": "4.13.0",
    "conda_location": "/opt/conda/lib/python3.9/site-packages/conda",
    "conda_prefix": "/opt/conda",
    "conda_private": false,
    "conda_shlvl": 2,
    "conda_version": "4.13.0",
    "config_files": [],
    "default_prefix": "/opt/conda/envs/birdy",
    "env_vars": {
      "CIO_TEST": "<not set>",
      "CONDA_DEFAULT_ENV": "birdy",
      "CONDA_EXE": "/opt/conda/bin/conda",
      "CONDA_PREFIX": "/opt/conda/envs/birdy",
      "CONDA_PREFIX_1": "/opt/conda",
      "CONDA_PROMPT_MODIFIER": "(birdy) ",
      "CONDA_PYTHON_EXE": "/opt/conda/bin/python",
      "CONDA_ROOT": "/opt/conda",
      "CONDA_SHLVL": "2",
      "CURL_CA_BUNDLE": "<not set>",
      "PATH": "/opt/conda/envs/birdy/bin:/opt/conda/condabin:/opt/conda/envs/birdy/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "REQUESTS_CA_BUNDLE": "<not set>",
      "SSL_CERT_FILE": "<not set>"
    },
    "envs": [],
    "envs_dirs": [
      "/opt/conda/envs",
      "/notebook_dir/writable-workspace/.home/.conda/envs"
    ],
    "netrc_file": null,
    "offline": false,
    "pkgs_dirs": [
      "/opt/conda/pkgs",
      "/notebook_dir/writable-workspace/.home/.conda/pkgs"
    ],
    "platform": "linux-64",
    "python_version": "3.9.12.final.0",
    "rc_path": "/notebook_dir/writable-workspace/.home/.condarc",
    "requests_version": "2.27.1",
    "root_prefix": "/opt/conda",
    "root_writable": true,
    "site_dirs": [],
    "sys.executable": "/opt/conda/bin/python",
    "sys.prefix": "/opt/conda",
    "sys.version": "3.9.12 (main, Apr  5 2022, 06:56:58) \n[GCC 7.5.0]",
    "sys_rc_path": "/opt/conda/.condarc",
    "user_agent": "conda/4.13.0 requests/2.27.1 CPython/3.9.12 Linux/4.18.0-348.20.1.el8_5.x86_64 debian/11 glibc/2.31",
    "user_rc_path": "/notebook_dir/writable-workspace/.home/.condarc",
    "virtual_pkgs": [
      [
        "__linux",
        "4.18.0",
        "0"
      ],
      [
        "__glibc",
        "2.31",
        "0"
      ],
      [
        "__unix",
        "0",
        "0"
      ],
      [
        "__archspec",
        "1",
        "x86_64"
      ]
    ]
  },
  "error": "RuntimeError(\"Operation not permitted: '/opt/conda/pkgs/cache/3e39a7aa.json'\")",
  "exception_name": "RuntimeError",
  "exception_type": "<class 'RuntimeError'>",
  "traceback": "Traceback (most recent call last):\n  File \"/opt/conda/lib/python3.9/site-packages/conda/exceptions.py\", line 1114, in __call__\n    return func(*args, **kwargs)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 935, in exception_converter\n    raise e\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 929, in exception_converter\n    exit_code = _wrapped_main(*args, **kwargs)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 887, in _wrapped_main\n    result = do_call(args, p)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 760, in do_call\n    exit_code = repoquery(args, parser)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/mamba.py\", line 702, in repoquery\n    pool = repoquery_api.create_pool(channels, platform, use_installed)\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/repoquery.py\", line 46, in create_pool\n    load_channels(\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/utils.py\", line 129, in load_channels\n    index = get_index(\n  File \"/opt/conda/lib/python3.9/site-packages/mamba/utils.py\", line 110, in get_index\n    is_downloaded = dlist.download(api.MAMBA_DOWNLOAD_FAILFAST)\nRuntimeError: Operation not permitted: '/opt/conda/pkgs/cache/3e39a7aa.json'\n"
}

[tlvu]

"error": "RuntimeError("Operation not permitted: '/opt/conda/pkgs/cache/3e39a7aa.json'")",

But I do have write access to that file, I tried echo >> /opt/conda/pkgs/cache/3e39a7aa.json and it works fine.

[tlvu]

@fcollonval I found it, the current user have to be the owner of /opt/conda/pkgs/cache/ folder. Just having write-access is not enough for mamba. I was enough for conda.

Should I still keep this issue open to fix the "current user have to be the owner of /opt/conda/pkgs/cache/ folder" to be equivalent with conda? If this won't be fixed then I can close this issue.

[fcollonval]

Thanks a lot @tlvu to dig that one up. Would you mind opening an issue in the mamba repository?

[tlvu]

Thanks a lot @tlvu to dig that one up. Would you mind opening an issue in the mamba repository?

Sure. Will close this issue once the other one in mamba repo opened.

[fcollonval]

Thanks