Use mamba rather than conda on the conda-CI

[mwouts]

Mamba is a faster replacement for conda. And it is indeed much faster! Locally the jupytext-ci environment is created in only 13s, while with conda it takes 1m45.

I'd like to use mamba on our conda CI, as this is now supported by the setup-miniconda action, cf. https://github.com/goanpeca/setup-miniconda#example-6-mamba

My first attempt https://github.com/mwouts/jupytext/commit/8961227c723ab06612ba19de8584c0754055ada6

• worked for Ubuntu / Python 2.7, and was indeed significantly faster (5m29) than the last run with conda (8m42s)
• but failed for Windows / Python 3.7, see the logs, or the extract below:

2020-10-08T09:01:33.2404174Z [command]C:\windows\system32\cmd.exe /D /S /C "C:\Miniconda\condabin\mamba.bat install --name jupytext-ci "python=3.7""
2020-10-08T09:01:33.7642262Z pkgs/main/win-64         Using cache
2020-10-08T09:01:33.7651258Z pkgs/main/noarch         Using cache
2020-10-08T09:01:33.7660264Z pkgs/r/win-64            Using cache
2020-10-08T09:01:33.7670560Z pkgs/r/noarch            Using cache
2020-10-08T09:01:33.7681821Z pkgs/msys2/win-64        Using cache
2020-10-08T09:01:33.7691694Z pkgs/msys2/noarch        Using cache
2020-10-08T09:01:33.7702582Z conda-forge/win-64       Using cache
2020-10-08T09:01:33.7713522Z conda-forge/noarch       Using cache
2020-10-08T09:01:35.5431597Z Transaction
2020-10-08T09:01:35.5434344Z 
2020-10-08T09:01:35.5437617Z   Prefix: C:\Miniconda\envs\jupytext-ci
2020-10-08T09:01:35.5445699Z 
2020-10-08T09:01:35.5452687Z   Updating specs:
2020-10-08T09:01:35.5454293Z 
2020-10-08T09:01:35.5457899Z    - python==3.7
2020-10-08T09:01:35.5458443Z 
2020-10-08T09:01:35.5458598Z 
2020-10-08T09:01:36.5835996Z ##[error]The process 'C:\Miniconda\condabin\mamba.bat' failed with exit code 3221226505

[mwouts]

@goanpeca, @wolfv, would you have any advice on how to get mamba working on Windows? Thanks!

[goanpeca]

There are indeed some issues on windows. Not sure if it is something we can fix on the action side. Thoughts @wolfv?

[wolfv]

yeah the windows issues on github actions are weird. I have tried to look into it before but haven't found any obvious solutions. I have no problems on Windows locally and we don't have any open bug reports on win-specific issues ...

[wolfv]

I think there are/will be fewer (of these kind of) issues with micromamba. which is already usable on Windows but it also has some outstanding issues that should be resolved before advising it's widespread usage.

[wolfv]

@dhirschfeld do you have any ideas? Could it be possible that mamba picks up dll's with the wrong/different versions (e.g. from chocolatey or nuget or whatever?).

[mwouts]

Thank you @wolfv for your comments. I can wait for micromamba, that's perfectly fine for me (and I am looking towards using it!). Is there some thread I can subscribe to to get an notification when it's ready for a larger usage? Thank you for working on this.

[wolfv]

I have a working micromamba ci example here: https://github.com/wolfv/multisheller/blob/main/.github/workflows/ci.yaml

We have to fix a small thing so that we can also use the magic link to always get the latest version from Windows (Powershell).

[goanpeca]

Hi @wolfv, is there some ENV var to set to have a larger/more complete debug for mamba, to try to isoalte the issue?

[dhirschfeld]

@dhirschfeld do you have any ideas?

Not really, sorry - these issues can be pretty inscrutable 😬

There seems to be some noise about broken VMWare tools that should be fixed with the latest version. But that doesn't help much as you don't control the host platform (and I'd be surprised if GitHub weren't using Hyper-V for virtualization) https://github.com/moby/moby/issues/40328#issuecomment-619854929

Otherwise, it's apparently a generic error so there could be other causes;

3221226505 looks to be a generic "Internal process error" coming from Windows.

https://github.com/moby/moby/issues/40328#issuecomment-569234487

[dhirschfeld]

Apparently it's a STATUS_STACK_BUFFER_OVERRUN https://vstinner.readthedocs.io/windows.html#windows-exceptions