mamba installer error "temp_directory_path" on Linux

[scottwsides]

I'm using Miniconda.....sh script to create conda python environment and I've conda installed mamba. Then any mamba install command gives

[swsides@haswell chemistream-all]$ cs-env/bin/mamba install -n base -y -c conda-forge scipy

              __    __    __    __
             /  \  /  \  /  \  /  \
            /    \/    \/    \/    \

███████████████/ /██/ /██/ /██/ /████████████████████████ / / \ / \ / \ / \ ____ / / _/ _/ _/ \ o _, / / ___/ ` |/ ███╗ ███╗ █████╗ ███╗ ███╗██████╗ █████╗ ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗ ██╔████╔██║███████║██╔████╔██║██████╔╝███████║ ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║ ██║ ╚═╝ ██║██║ ██║██║ ╚═╝ ██║██████╔╝██║ ██║ ╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝

    mamba (0.27.0) supported by @QuantStack

    GitHub:  https://github.com/mamba-org/mamba
    Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████

Looking for: ['scipy']

filesystem error: temp_directory_path: No such file or directory

[jonashaag]

Not sure if it’s related but you should not install anything into the base env.

[scottwsides]

But why? It seems to work fine on windows, mac and linux ... before switching to mamba

[jonashaag]

It can mess up your Conda installation. Conda lives in the base environment.

[wolfv]

@scottwsides Hmm, that looks like a slightly hacky way of doing things, I am afraid. If I understand correctly, you installed mamba into cs-env which you are calling without activating the env?

Mamba depends on conda so they should live side-by-side in the base environment (to benefit from a shared package cache and so on).

The error likely comes from different versions of libstdc++. Probably your (activated) base environment uses the libstdc++ from defaults, while the cs-env ships the proper version from conda-forge. Since you don't activate the env it doesn't find the proper symbols.

If you activate the environment, conda & mamba might think that the base environment is cs-env, by the way. That's why it's suggested to install mamba in the base env.

Anyways, it's even easier to start with mambaforge or micromamba.

[scottwsides]

Sorry no. cs-env is simply the directory where I have conda installed (from the Miniconda script). I put everything into the conda base env. My mamba is in the base eg (conda install --name base -y -c conda-forge mamba).

But... perhaps I should start with micromamba. Where is the 'drop-in' replacement for Miniconda....sh?

I've tried mamba installs on linux after rebooting machine and updating miniconda version. Still getting temp_directory_path install error

[jonashaag]

https://mamba.readthedocs.io/en/latest/installation.html#micromamba

[scottwsides]

Tried micromamba and I'm still getting the following error

critical libmamba Multiple errors occured: filesystem error: temp_directory_path: No such file or directory filesystem error: temp_directory_path: No such file or directory

Is this my machine? (i've rebooted already) I've seen this on another linux machine

[jonashaag]

Can you try this on the machine?

// tmp.cpp
#include <iostream>
#include <filesystem>

int main() {
  std::cout << std::filesystem::temp_directory_path().c_str();
}

c++ -std=c++17 tmp.cpp -o tmp && file $(./tmp)

[scottwsides]

[swsides@haswell scr_haswell]$ c++ -std=c++17 tmp.cpp -o tmp && file $(./tmp) /tmp/ccV30f49.o: In function main': tmp.cpp:(.text+0x11): undefined reference to std::filesystem::temp_directory_pathabi:cxx11' collect2: error: ld returned 1 exit status

Hmmm... I suppose compilers need to be updated. I think there are other posts about c++17 being a little un-reasonable to require. Whats the status of that?

[jonashaag]

Just FYI editing a post doesn't send any notifications so I didn't see your question.

I think most compilers younger than 5 years old do support C++17 so I don't think it's unreasonable.

You can try this with C++11:

// tmp.cpp
#include <iostream>
#include <filesystem>

int main() {
  std::cout << std::__fs::filesystem::temp_directory_path().c_str();
}

[wolfv]

I think the problem is that the STL of C++ is not statically linked in and you have an older one that doesn't have the filesystem symbols :/ maybe @JohanMabille knows something about this.

[wolfv]

Or @Klaim actually

[scottwsides]

Yes I was on a machine with an old compiler. Moving to new machine cleared up the problem. Sorry to bother

[hannah413]

I'm having the same issue from within a non-base environment. On a cluster. Any recommendations on what to tell my sysadmins to resolve this issue?

[wolfv]

I think we should link statically with libc++ (thanks to @isuruf we're already doing that on macOS)

[isuruf]

micromamba links statically to libstdc++ on Linux, so that's not the issue

[jonashaag]

@hannah413 can you try my test program please?

[jonashaag]

Create a file tmp.cpp with the contents in the first code block and run the second code block in bash.

[hannah413]

hold on--learning as i go. renamed the file and ran with the bash line. deleting unhelpful comment above.

here is the error I run into now when I run your bash line after correctly naming the cpp file.

c++: error: unrecognized command line option ‘-std=c++17’

[jonashaag]

Uh oh, that seems to be a very old C++ compiler that you have installed. To get the version:

c++ --version
# Or maybe this is more recent? Usually it's the same compiler
g++ --version

Do you have any way to use a more recent C++ compiler?

[hannah413]

lol keep shaming me here and maybe the guys in our computing center that i've linked to this thread will do something about it:

c++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)

and same for g++

i don't think i have a way to use a more recent c++ compiler on this cluster. i could do it locally but i tend to employ conda (and will be employing mamba) for large parallel tasks.

[jonashaag]

Actually you could try the C++ compiler from conda-forge:

conda install -c conda-forge cxx-compiler

[jonashaag]

University clusters tend to have ancient versions of just about everything for reasons that I don’t understand. Best to use conda-forge heavily :-)

[hannah413]

ok, successfully got cxx-compiler, now we're here:

(testmamba) [holly@xxx2 holly]$ c++ -std=c++17 testmamba.cpp -o tmp && file $(./tmp)

testmamba.cpp: In function 'int main()':
testmamba.cpp:6:21: error: 'std::__fs' has not been declared
    6 |   std::cout << std::__fs::filesystem::temp_directory_path().c_str();
      |                     ^~~~

considering going this route but i might as well be a monkey with a wrench right now as this is out of my domain: https://stackoverflow.com/questions/55474690/stdfilesystem-has-not-been-declared-after-including-experimental-filesystem

[jonashaag]

You can remove __fs:: as in my first snippet, it was added only to work with an older compiler (although not as old as the one on your cluster :-D) https://github.com/mamba-org/mamba/issues/2042#issuecomment-1295267591

[hannah413]

Thanks! I'm still getting a temp_directory_path error:

(testmamba) [holly@xxx2 holly]$ c++ -std=c++17 testmamba.cpp -o tmp && file $(./tmp)

terminate called after throwing an instance of 'std::filesystem::__cxx11::filesystem_error'
  what():  filesystem error: temp_directory_path: No such file or directory
Usage: file [-bchikLlNnprsvz0] [--apple] [--mime-encoding] [--mime-type]
  [-e testname] [-F separator] [-f namefile] [-m magicfiles] file ...
       file -C [-m magicfiles]
       file [--help]

[jonashaag]

That’s really interesting. You used a recent compiler but the problem remains. I’m not sure what happens here. Are we getting an old libstdc++ even with the new compiler? (Because it’s dynamically linked and using the system libstdc++?) And that version doesn’t support that temporarily directory API? If so it means we are in fact not statically linking libstdc++ right? @isuruf @wolfv

[isuruf]

@jonashaag, I don't know why you are thinking this is a libstdc++ error. If the issue is a libstdc++ version issue, then the executable will fail with a missing symbol error, which is clearly not the case here.

@hannah413, do you have any of the environment variables TMPDIR, TMP, TEMP, TEMPDIR set?

[Klaim]

I believe -lstdc++fs is missing in the flags, its not necessary after gcc8 but if I understand this is an older gcc than 8?

[jonashaag]

@isuruf my thinking is that when we fixed the compilation problems on @hannah413‘s machine we got the program to compile but it fails to find the temporary directory. My hypothesis was that this is due to an old version of libstdc++ that has a buggy implementation of that API but you are right that it could also be due to weird environment, eg. missing TMP environment variables.