datatable
datatable copied to clipboard
h2oai
Provide ARM64 wheels
EDIT: It looks like the development version builds without issue, e.g.
$ pip install git+https://github.com/h2oai/datatable
Collecting git+https://github.com/h2oai/datatable
Cloning https://github.com/h2oai/datatable to /private/var/folders/9n/3nxsgkrj0rlfx196kzdttpww0000gn/T/pip-req-build-f00uim_e
Running command git clone --filter=blob:none -q https://github.com/h2oai/datatable /private/var/folders/9n/3nxsgkrj0rlfx196kzdttpww0000gn/T/pip-req-build-f00uim_e
Resolved https://github.com/h2oai/datatable to commit d54d7f21110ea1f19f70a57034d14feeea6e3aaf
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: datatable
Building wheel for datatable (pyproject.toml) ... done
Created wheel for datatable: filename=datatable-1.1.0a0+build.1641510381.kevin-cp39-cp39-macosx_12_arm64.whl size=7390850 sha256=441a4adb83b3e86a03c6ad9f1a508c934ebe04e8d6e07294995a4959da69d784
Stored in directory: /private/var/folders/9n/3nxsgkrj0rlfx196kzdttpww0000gn/T/pip-ephem-wheel-cache-725mfy2l/wheels/23/f2/aa/36775b914ff0f3b74fe9765aeaa0a958db373dd6bd61186923
Successfully built datatable
Installing collected packages: datatable
Successfully installed datatable-1.1.0a0+build.1641510381.kevin
Are there plans to update datatable on PyPI soon?
I saw the following when trying to install datatable via pip:
Start setup.py
command = `install`
setuptools version = 59.0.1
Find an LLVM installation
Environment variable LLVM is not set
Environment variable LLVM7 is not set
Environment variable LLVM6 is not set
Environment variable LLVM5 is not set
Environment variable LLVM4 is not set
Candidate directory `/usr/local/opt/llvm` not found
The build will proceed without Llvm support
Determine the compiler
Environment variable `CXX` is not set
Environment variable `CC` is not set
Llvm installation not found, cannot search for the clang++ compiler
distutils.ccompiler reports the default compiler to be `cc`
Compiler `clang++` will be used
clang++ --version :
Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: arm64-apple-darwin21.2.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Prepare the environment
CC = clang++
CXX = clang++
LDFLAGS =
ARCHFLAGS = -m64
LLVM_CONFIG =
MACOSX_DEPLOYMENT_TARGET = 10.13
Find compile include directories
`c` is the main C++ source directory
`/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9` added from CONFINCLUDEPY
Determine the extra compiler flags
-std=c++11
-stdlib=libc++
-Ic
-fPIC
-g2
-O3
-Weverything
-Wno-c++98-compat-pedantic
-Wno-c99-extensions
-Wno-exit-time-destructors
-Wno-float-equal
-Wno-global-constructors
-Wno-reserved-id-macro
-Wno-switch-enum
-Wno-weak-template-vtables
-Wno-weak-vtables
-Ic
-I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9
-isystem /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9
Find the required dynamic libraries
Library `libc++.1.dylib` found at /opt/homebrew/Cellar/llvm/13.0.0_2/lib/libc++.1.dylib
Library `libc++abi.dylib` found at /opt/homebrew/Cellar/llvm/13.0.0_2/lib/libc++abi.dylib
Determine the extra linker flags
-Wl,-rpath,@loader_path/.
-L/opt/homebrew/Cellar/llvm/13.0.0_2/lib
Copy dynamic libraries
Copying /opt/homebrew/Cellar/llvm/13.0.0_2/lib/libc++.1.dylib to datatable/lib/libc++.1.dylib
Copying /opt/homebrew/Cellar/llvm/13.0.0_2/lib/libc++abi.dylib to datatable/lib/libc++abi.dylib
Generate git version file (force)
Environment variable DTBL_GIT_HASH not present
Source distribution detected, file __git__.py cannot be rebuilt
Determine datatable version
Reading file /private/var/folders/9n/3nxsgkrj0rlfx196kzdttpww0000gn/T/pip-install-qf7bxo04/datatable_63f9e18f4c284db4a4980445527d3f5d/datatable/__version__.py
Detected version: 0.10.1
Environment variable CI_VERSION_SUUFFIX not present
< ... other compilation output ... >
Post-link processing
Output file: build/lib.macosx-12-arm64-3.9/_datatable.cpython-39-darwin.so
Checking dependencies of _datatable.cpython-39-darwin.so
$ otool -L build/lib.macosx-12-arm64-3.9/_datatable.cpython-39-darwin.so
/opt/homebrew/opt/llvm/lib/libc++.1.dylib
/usr/lib/libSystem.B.dylib
Relocating dependency libc++.1.dylib
$ install_name_tool -change /opt/homebrew/opt/llvm/lib/libc++.1.dylib @rpath/libc++.1.dylib build/lib.macosx-12-arm64-3.9/_datatable.cpython-39-darwin.so
Checking dependencies of _datatable.cpython-39-darwin.so
$ otool -L build/lib.macosx-12-arm64-3.9/_datatable.cpython-39-darwin.so
@rpath/libc++.1.dylib
/usr/lib/libSystem.B.dylib
Checking dependencies of libc++.1.dylib
$ otool -L datatable/lib/libc++.1.dylib
/usr/lib/libSystem.B.dylib
@rpath/libc++abi.1.dylib
Dependency datatable/lib/libc++abi.1.dylib does not exist
----------------------------------------
Any idea, or anything else I can provide to help debug? This is on an M1 macOS machine, compiling for ARM64.
The only thing that stands out to me is that libc++abi.dylib is found:
Find the required dynamic libraries
Library `libc++.1.dylib` found at /opt/homebrew/Cellar/llvm/13.0.0_2/lib/libc++.1.dylib
Library `libc++abi.dylib` found at /opt/homebrew/Cellar/llvm/13.0.0_2/lib/libc++abi.dylib
But the link / install_name_tool step seems to want libc++abi.1.dylib:
Dependency datatable/lib/libc++abi.1.dylib does not exist
What is the version of pip you're using? We recommend using 20.3+.
For macOS it shouldn't start building datatable from source, but rather install it from the wheel. Try upgrading your pip to the latest version.
$ pip --version
pip 21.3.1 from /Users/kevin/.virtualenvs/r-reticulate/lib/python3.9/site-packages/pip (python 3.9)
If I understand correctly, there aren't any ARM64 wheels of datatable available (https://pypi.org/project/datatable/#files) so pip falls back to installing from sources.
I see. Yes, we do not provide wheels for ARM64. I remember we had similar (but not exactly the same issues) on M1: https://github.com/h2oai/datatable/issues/3003
Take a look at https://github.com/h2oai/datatable/issues/3003#issuecomment-866387886 and see if it helps.
I updated the issue to indicate that the issue seems to be resolved with the development version of datatable, so I guess this is ultimately a request (for users on M1 macOS) to have a new version of datatable released to PyPI, with the associated ARM64 binaries if possible. Thanks!
I see, thanks. Unfortunately, for the moment we don't have M1 machines in our possession. Once we have an option, we will start providing ARM64 binaries.
Hi guys, I guess usually what you would do with an M1 Mac is to get the macOS 64-bit universal2 installer. If you then attempt to install datatable it leads to the issue above or something similar. A quick and dirty solution is to get the macOS 64-bit Intel-only installer (I know not something you would want to do with your new M1 machine). However, you can install datatable without any issues. It might be a strange solution but it's super easy! (https://www.python.org/downloads/macos/)
Irrespective of that, I think the whole Apple Silicon user base of datatable would highly appreciate direct compatibility.
Hi guys, I was able to build a wheel on my Mac M1 following the below steps with the help of conda environment
Make sure you have anaconda/miniconda installed
git clone git clone https://github.com/h2oai/datatable.git
cd datatable
conda create --name datatable38 python=3.8
conda activate datatable38
make wheel
pip install dist/datatable-1.1.0a0+build.1644259735.mathanraj-cp38-cp38-macosx_10_9_x86_64.whl
Hope this helps ...
