jupyenv
jupyenv copied to clipboard
tweag
Unable to install on MacOS
Describe the bug I can't install jupyterWith on MacOS 11.1. Everything else with nix looks to work just fine.
To Reproduce Use the example shell.nix on MacOS and run shell-nix. I get the following errors:
======================================================================
ERROR: Failure: AttributeError (dlsym(RTLD_DEFAULT, GetMacOSStatusCommentString): symbol not found)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/nix/store/hwq3sz718glihjsnzfg426jzfcd7cavn-python3.7-nose-1.3.7/lib/python3.7/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/nix/store/hwq3sz718glihjsnzfg426jzfcd7cavn-python3.7-nose-1.3.7/lib/python3.7/site-packages/nose/loader.py", line 417, in loadTestsFromName
addr.filename, addr.module)
File "/nix/store/hwq3sz718glihjsnzfg426jzfcd7cavn-python3.7-nose-1.3.7/lib/python3.7/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/nix/store/hwq3sz718glihjsnzfg426jzfcd7cavn-python3.7-nose-1.3.7/lib/python3.7/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/nix/store/mifia80k0frxaq68c7g1m3apmdz47f9j-python3-3.7.7/lib/python3.7/imp.py", line 234, in load_module
return load_source(name, filename, file)
File "/nix/store/mifia80k0frxaq68c7g1m3apmdz47f9j-python3-3.7.7/lib/python3.7/imp.py", line 171, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 696, in _load
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/private/tmp/nix-build-python3.7-notebook-6.0.3.drv-0/notebook-6.0.3/notebook/auth/tests/test_login.py", line 6, in <module>
from notebook.tests.launchnotebook import NotebookTestBase
File "/private/tmp/nix-build-python3.7-notebook-6.0.3.drv-0/notebook-6.0.3/notebook/tests/launchnotebook.py", line 24, in <module>
from ..notebookapp import NotebookApp
File "/private/tmp/nix-build-python3.7-notebook-6.0.3.drv-0/notebook-6.0.3/notebook/notebookapp.py", line 81, in <module>
from .services.contents.filemanager import FileContentsManager
File "/private/tmp/nix-build-python3.7-notebook-6.0.3.drv-0/notebook-6.0.3/notebook/services/contents/filemanager.py", line 17, in <module>
from send2trash import send2trash
File "/nix/store/nq57qib2r35f8k38b310gndsiadckw2x-python3.7-Send2Trash-1.5.0/lib/python3.7/site-packages/send2trash/__init__.py", line 12, in <module>
from .plat_osx import send2trash
File "/nix/store/nq57qib2r35f8k38b310gndsiadckw2x-python3.7-Send2Trash-1.5.0/lib/python3.7/site-packages/send2trash/plat_osx.py", line 17, in <module>
GetMacOSStatusCommentString = Foundation.GetMacOSStatusCommentString
File "/nix/store/mifia80k0frxaq68c7g1m3apmdz47f9j-python3-3.7.7/lib/python3.7/ctypes/__init__.py", line 377, in __getattr__
func = self.__getitem__(name)
File "/nix/store/mifia80k0frxaq68c7g1m3apmdz47f9j-python3-3.7.7/lib/python3.7/ctypes/__init__.py", line 382, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, GetMacOSStatusCommentString): symbol not found
Expected behavior The installation instructions should work as expected on MacOS.
Environment
- OS name + version: MacOS 11.1
- Version of the code: revision 35eb565c6d00f3c61ef5e74e7e41870cfa3926f7o
Additional context Looks like to be related to https://github.com/arsenetar/send2trash/issues/51
The update to Big Sur broke lots of packages. There is a fix in a newer version of Nixpkgs, but I will have to update it in this repo, and it always takes some work to put things back in shape. I will try to schedule some time to do it soon. In the meantime, you can try using a different snapshot yourself.
@guaraqe Thanks!
Can you let me know how to use a different snapshot? I have already updated my nixpkgs snapshot (I use flakes and nixpkgs unstable). Do I need to point to a different pkgs when using jupyterWith?
This is the entrypoint for setting Nixpkgs: https://github.com/tweag/jupyterWith/blob/35eb565c6d00f3c61ef5e74e7e41870cfa3926f7/default.nix#L3
But since there are many overrides, setting it may not be a very smooth experience.
@guaraqe I'm going to PR the bumpup version of nixpkgs. Do you have time to review it?
This is still broken for me using a flake. I saw that nixpkgs uses 3.9 which supposedly has a fix for this, but it still fails. I've also tried using the flake example and it does not properly build on macOS 11.6.
The `flake.nix` I'm using
{
description = "Flake for developing sara";
nixConfig.extra-substituters = "https://jupyterwith.cachix.org";
nixConfig.extra-trusted-public-keys = "jupyterwith.cachix.org-1:/kDy2B6YEhXGJuNguG1qyqIodMyO4w8KwWH4/vAc7CI=";
inputs = {
flake-utils.url = "github:numtide/flake-utils";
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
# nixpkgs.url = "nixpkgs/a5d03577f0161c8a6e713b928ca44d9b3feb2c37";
mach-nix.url = "github:DavHau/mach-nix/3.3.0";
jupyterWith.url = "github:tweag/jupyterWith/master";
};
# https://github.com/DavHau/mach-nix/issues/153
outputs = inputs@{ self, nixpkgs, flake-utils, mach-nix, jupyterWith }:
(flake-utils.lib.eachSystem [ "x86_64-linux" "x86_64-darwin" ] (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = nixpkgs.lib.attrValues jupyterWith.overlays
++ [ self.overlay ];
config = {
allowBroken = true;
allowUnfree = true;
allowUnsupportedSystem = true;
};
};
in rec {
devShell = with pkgs;
let
iPython = pkgs.jupyterWith.kernels.iPythonWith {
name = "Python-data-env";
ignoreCollisions = true;
};
jupyterEnvironment = pkgs.jupyterWith.jupyterlabWith {
kernels = [ iPython ];
directory = "./.jupyterlab";
};
in pkgs.mkShell rec {
buildInputs = [ jupyterEnvironment ];
shellHook = "";
};
})) // {
overlay = final: prev: { };
};
}
Most recent output
[1] cportela@mac> nix develop ~/src/proj
warning: Git tree '/Users/cportela/src/proj' is dirty
warning: updating lock file '/Users/cportela/src/proj/flake.lock':
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/a5d03577f0161c8a6e713b928ca44d9b3feb2c37' (2021-06-11)
→ 'github:NixOS/nixpkgs/ee084c02040e864eeeb4cf4f8538d92f7c675671' (2021-10-04)
warning: Git tree '/Users/cportela/src/proj' is dirty
error: builder for '/nix/store/ldgpcnbli1xpvyfplahs64gsw735aiih-python3.9-jupyter_server-1.11.0.drv' failed with exit code 1;
last 10 log lines:
> from send2trash import send2trash
> File "/nix/store/b18s65qwy9z006h3z46v9rwjcalhw0c1-python3.9-Send2Trash-1.5.0/lib/python3.9/site-packages/send2trash/__init__.py", line 12, in <module>
> from .plat_osx import send2trash
> File "/nix/store/b18s65qwy9z006h3z46v9rwjcalhw0c1-python3.9-Send2Trash-1.5.0/lib/python3.9/site-packages/send2trash/plat_osx.py", line 17, in <module>
> GetMacOSStatusCommentString = Foundation.GetMacOSStatusCommentString
> File "/nix/store/dwwpb9hm9925grl9kc28fc0phc4qvips-python3-3.9.6/lib/python3.9/ctypes/__init__.py", line 387, in __getattr__
> func = self.__getitem__(name)
> File "/nix/store/dwwpb9hm9925grl9kc28fc0phc4qvips-python3-3.9.6/lib/python3.9/ctypes/__init__.py", line 392, in __getitem__
> func = self._FuncPtr((name_or_ordinal, self))
> AttributeError: dlsym(RTLD_DEFAULT, GetMacOSStatusCommentString): symbol not found
For full logs, run 'nix log /nix/store/ldgpcnbli1xpvyfplahs64gsw735aiih-python3.9-jupyter_server-1.11.0.drv'.
error: 1 dependencies of derivation '/nix/store/i38h406hdj7la1pn0b12y8b7q40y6dpa-python3.9-jupyterlab_server-2.8.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/81bzsg3y5l6mn6qcvlxqdkb58h732kg5-python3.9-jupyterlab-3.1.14.drv' failed to build
error: 1 dependencies of derivation '/nix/store/q185ash0la3q2m9iv4arl87m0cz0q7fc-nix-shell-env.drv' failed to build
I've seen in other issues that some fixes have been merged, but either I'm using them wrong or something is missing. I can also see the Darwin builds (non-Flake) are passing on master so I'm confused as to what the issue is and why what I have isn't working. From what I can tell I'm doing the same thing as the example and I'm using the later version of nixpkgs which should have the fixes for this (I thought).
The Flake is currently only working on Linux, we didn't manage making it working on MacOS. However, the standard way of using JupyterWith should be working.
using JupyterWith should be working
there is nothing different between flake and standard way.
Do not updating lock file. we are using the same rev of nixpkgs with that.
or using https://github.com/NixOS/nixpkgs/pull/136254 staging branch instead.
- solution: https://github.com/NixOS/nixpkgs/issues/136485#issuecomment-913760454
then, overriding the python3 to python3.8.12.
finally, please run
nix develop --impuretry again.
see https://github.com/NixOS/nixpkgs/issues/136485#issuecomment-988666141 for an alternative quick-fix
just set the environment variable first
nix-shell --pure -p python39Packages.jupyter
export DYLD_LIBRARY_PATH=/System/Library/Frameworks/Foundation.framework/Versions/C/Resources/BridgeSupport
jupyter notebook
It is due to a bug in python's ctypes.util.find_library().
@larsr thanks very much. it could be solved at https://github.com/tweag/jupyterWith/pull/179.
The PR was merged on master. If you use the commit recommended in the README, it should (hopefully) work.
@guaraqe I noticed that the patch modifies send2trash, but the problem is really in ctypes.
While I do see the usefulness of not touching a central package like ctypes, patching a 'correct' package does not feel like the right solution, at least not long term.
So that patch should probably be reverted once (if ever) the corrected ctypes (which is in python 3.9.7 and later ) reaches nix. Just something to keep in mind...
For next version of jupyterWith (see main branch) we have on our roadmap to get better support (or at least document what doesn't work) for MacOS.
