CaImAn
CaImAn copied to clipboard
flatironinstitute
Can't install caimanmanager.py in macOS
For better support, please use the template below to submit your issue. When your issue gets resolved please remember to close it.
Sometimes errors while running CNMF occur during parallel processing which prevents the log to provide a meaningful error message. Please reproduce your error with setting dview=None.
If you need to upgrade CaImAn follow the instructions given in the documentation.
- Tell us a bit about your setup:
-
Operating system (Linux/macOS/Windows): macOS 11.2.1 (Apple M1)
-
Python version (3.x): 3.7.9
-
Working environment (Python IDE/Jupyter Notebook/other): terminal
-
Which of the demo scripts you're using for your analysis (if applicable):
-
CaImAn version*: not sure which version, but I just downloaded it on 2/18/2021
-
CaImAn installation process (
pip install ./pip install -e ./conda): conda-forge package
*You can get the CaImAn version by creating a params object and then typing params.data['caiman_version']. If the field doesn't exist, type N/A and consider upgrading)
- Describe the issue that you are experiencing I want to install caiman to a new MacBook. But after running Conda-forge command and successfully installed caiman, I went to my data directory (i.e. not anaconda3/env/caiman) and run "caimanmanager.py install" but failed (Error message see below). Other things I tried are as follows:
- "python caimanmanager.py install"
- "caimanmanager.py help" (same error message)
- "python /Users/shunli/opt/anaconda3/env/caiman/bin/caimanmanager.py install" (same error)
I'm suspecting that the program is running but failed at "from caiman.paths import caiman_datadir" (see attached figures
).
-
Copy error log below (base) shunli@Shuns-MacBook-Air ~ % conda activate caiman
(caiman) shunli@Shuns-MacBook-Air ~ % cd Desktop/McHenry\ Lab/CaImAn (caiman) shunli@Shuns-MacBook-Air CaImAn % caimanmanager.py install
zsh: illegal hardware instruction caimanmanager.py install -
If you're not reporting an error, type your message below
Right now we don't have any test hardware for the new ARM-based Macs, and it's likely that the new platform will run into issues like this. As soon as ARM-based systems become available that have a reasonable amount of RAM I'll be ordering one so I can dig into issues like this.
Note that because of the limited RAM available on all presently purchasable ARM-based Mac systems, even after we solve the issues you may find it challenging to work with a lot of data sets.. Although it will remain a good way to build familiarity with the codebase.
@shunnnli Giving you an update on this: Conda support for arm64 is arriving, but it's taking awhile. I'm feeling hopeful seeing things like this:
https://conda-forge.org/blog/posts/2020-10-29-macos-arm64/
but we're still waiting for Tensorflow and a few other things to be ported over (tensorflow on osx has been a stumbling block for awhile).
Tensorflow is no longer an issue; there are now packages of tensorflow 2.4 linked against py38 for OSX on arm64 and x86.
Right now the only real blocker from my getting started on this is that the M1 chips have a common configuration of only 8G RAM (which conda itself struggles with building environments) and 16G as an option (which is just enough to squeak by with conda and caiman, but it's not comfortable).
If It's not an urgent need, I'd like to wait for a 32G ARM Mac to be released, although I don't know when that will be. I'm hoping the M1X or M2 CPU will be available in that configuration.
My new MacBook Pro with M1 chip and 16GB RAM arrived. The package-based as well as development mode install of CaImAn are both successful. The nosetests all succeed.
The package-based install is however much slower than development mode. The conda create -n caiman -c conda-forge caiman took 62 min, whereas conda env create -f environment.yml -n caiman took merely 7.5 min.
@pgunn should we update the readme's Requirements section accordingly?
Right now I'm thinking of changing our recommendations to use mamba rather than the baseline conda tools; I've had much more luck with making environments quickly with mamba.
My understanding is that if you did a package-based install of caiman on M1, you probably ended up with an x86 package (maybe the entire environment) and are going through the x86 emulator. I might be wrong on this; if you don't mind poking around in the package dir, the "file" command should be able to tell you what arch the compiled bits ended up being and I'd be interested to know.
I'm wary of advertising ARM support until we have osx-arm64 targeted packages, and until I have an arm dev box (I'm waiting for a 64G system to become available) as I won't be able to support user issues well until then.
That's indeed the case. In the package dir file python only outputs python: Mach-O 64-bit executable x86_64, whereas file /usr/bin/python yields
/usr/bin/python: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/python (for architecture x86_64): Mach-O 64-bit executable x86_64
/usr/bin/python (for architecture arm64e): Mach-O 64-bit executable arm64e
Fascinating. I wasn't aware they had fat binaries in the single-file form (I know bundles had it, but that's way easier).
This makes things both easier and more confusing - we can support x86 Caiman on ARM64 (although we should nudge people pretty hard away from trying on an 8G config), but at some point in the future we'll need to deal with cross-linking and making sure people don't end up with hybrid environments.
I'm not sure what to do. I know longer-term, the conda project intends to support ARM64 as an architecture. I have no idea how they'll avoid that being a big confusing mess.
An update on this: I now have one of the new ARM64 Macs with adequate ram; it's possible to make an ARM64 environment with conda now (installers were recently added), but enough packages have not yet been ported that we can't yet meet prerequisites in a pure arm64 environment.
I expect this to change within the coming months, and will keep an eye on it.
Hi, does anyone have an update on this? Would like to use CalmAn on a new mac workstation but concerned that it won't be compatible.
Hello @noamza For now, if you use a standard install of conda and use the mamba tool to install the caiman environment/package, everything should work fine (but please make sure you get one of the laptops with more RAM if you go down the ARM route).
@shunnnli If you're still seeing the issue, I suggest making sure you have the latest OSX updates and doing a fresh install of conda; if that doesn't fix it please reach out to me and I'll try to help you with that error you saw (but I suspect it was due to using early hardware/software.
Hi! I just got a brand new macbook m1 and have limited experience setting up virtual environments. should i use the mamba tool to install the caiman environment / package? i have 16g ram
Hello @bpn228 , it's probably best you don't reuse existing github issues unrelated to your questions.
Anyhow, it is possible to use Caiman on that system, but the 16G of RAM means you'll only be able to process fairly small datasets. If that's okay, then install miniconda, install mamba, and then use mamba to install caiman. Please reach out to me if you have any issues, either with a new issue or on gitter.
Best wishes, Pat
Hi I got error when I tried to install camlanmanager in ubuntu windows 11
Traceback (most recent call last):
File "/home/shuai/miniforge3/envs/mescore/bin/caimanmanager", line 7, in
@Shuai-NY
Hello,
It's generally best if you open a new ticket for install issues so the person who used the original ticket doesn't get all the notifications as we try to solve the problem; if you can do that, and also include the output of conda list opencv from your environment we can try to find the problem (on first guess this may be some missing GL systems packages with your Linux install).