unicorn
unicorn copied to clipboard
unicorn-engine
Install fails on Kali Arm64
Trying to install Unicorn on Kali Linux: Linux kali-linux-2021-3 5.14.0-kali2-arm64 #1 SMP Debian 5.14.9-2kali1 (2021-10-04) aarch64 GNU/Linux
Install Command
┌──(parallels㉿kali-linux-2021-3)-[~/Documents/sft/unicorn-1.0.3/build]
└─$ pip install unicorn
Error
Defaulting to user installation because normal site-packages is not writeable
Collecting unicorn
Using cached unicorn-2.0.0.tar.gz (2.7 MB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: unicorn
Building wheel for unicorn (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [4 lines of output]
running bdist_wheel
running build
Building C extensions
error: [Errno 2] No such file or directory: '/tmp/pip-install-8fr1v2ls/unicorn_f78886f8abfc407cb68bf41c94e7ee38/../../include/unicorn'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for unicorn
Running setup.py clean for unicorn
Failed to build unicorn
Installing collected packages: unicorn
Running setup.py install for unicorn ... error
error: subprocess-exited-with-error
× Running setup.py install for unicorn did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
running install
/home/parallels/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
Building C extensions
error: [Errno 2] No such file or directory: '/tmp/pip-install-8fr1v2ls/unicorn_f78886f8abfc407cb68bf41c94e7ee38/../../include/unicorn'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> unicorn
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
This would really streamline the install. How can I help out with this?
Has it been solved?
Not yet but I believe it's mostly CI stuff and easy to fix. Just too busy to test it.
@WizardsOfTheInternet, not a lot you can do to help without official PyPi upload privileges, but here's what unicorn does in CI, which you can reproduce locally, roughly:
sudo apt-get install python-venv
python3 -m venv venv && source activate venv
pip install wheel auditwheel
git pull https://github.com/unicorn-engine/ && cd unicorn
./bindings/python/build_wheel.sh
@wtdcode Or other maintainers would just modify CI accordingly to publish the appropriate wheel, it just takes some trial & error w/ CI and get it right/done/published on PyPi... ergo takes time. For instance, fixing fun stuff like this one:
(venv) ubuntu@home:~/galaxy_s4/unicorn$ ./bindings/python/build_wheel.sh
(...)
INFO:auditwheel.main_repair:Repairing unicorn-2.0.0-py2.py3-none-manylinux1_aarch64.whl
usage: auditwheel [-h] [-V] [-v] command ...
auditwheel: error: cannot repair "unicorn-2.0.0-py2.py3-none-manylinux1_aarch64.whl" to "manylinux_2_17_aarch64" ABI because of the presence of too-recent versioned symbols. You'll need to compile the wheel on an older toolchain.
But in fact once you have the wheel generated locally you can install it yourself right away.
Hope that helps somehow ;)
@WizardsOfTheInternet, not a lot you can do to help without PyPi privileges, but here's what unicorn does in CI, which you can reproduce locally, roughly:
sudo apt-get install python-venv python3 -m venv venv && source activate venv pip install wheel auditwheel git pull https://github.com/unicorn-engine/ && cd unicorn ./bindings/python/build_wheel.sh@wtdcode Or other maintainers would just modify CI accordingly to publish the appropriate wheel, it just takes some trial & error w/ CI and get it right/done/published on PyPi... ergo takes time. For instance, fixing fun stuff like this one:
(venv) ubuntu@home:~/galaxy_s4/unicorn$ ./bindings/python/build_wheel.sh (...) INFO:auditwheel.main_repair:Repairing unicorn-2.0.0-py2.py3-none-manylinux1_aarch64.whl usage: auditwheel [-h] [-V] [-v] command ... auditwheel: error: cannot repair "unicorn-2.0.0-py2.py3-none-manylinux1_aarch64.whl" to "manylinux_2_17_aarch64" ABI because of the presence of too-recent versioned symbols. You'll need to compile the wheel on an older toolchain.Hope that helps somehow ;)
You are almost on the right track. The error is that you should use an old enough Linux distribution to build the wheel. In most cases, you could find it here: https://github.com/pypa/manylinux
You are almost on the right track. The error is that you should use an old enough Linux distribution to build the wheel. In most cases, you could find it here: https://github.com/pypa/manylinux
Or go through another not so backwards compatible track:
diff --git a/bindings/python/build_wheel.sh b/bindings/python/build_wheel.sh
index e3c9d444..5429685d 100755
--- a/bindings/python/build_wheel.sh
+++ b/bindings/python/build_wheel.sh
@@ -10,5 +10,5 @@ else
python3 setup.py bdist_wheel
fi
cd dist
-auditwheel repair *.whl
-mv -f wheelhouse/*.whl .
+#auditwheel repair *.whl
+#mv -f wheelhouse/*.whl .
diff --git a/bindings/python/setup.py b/bindings/python/setup.py
index 51dfb9fe..20f1f4bd 100755
--- a/bindings/python/setup.py
+++ b/bindings/python/setup.py
@@ -177,7 +177,7 @@ if 'bdist_wheel' in sys.argv and '--plat-name' not in sys.argv:
idx = sys.argv.index('bdist_wheel') + 1
sys.argv.insert(idx, '--plat-name')
name = get_platform()
- if 'linux' in name:
+ if 'boo' in name:
# linux_* platform tags are disallowed because the python ecosystem is fubar
# linux builds should be built in the centos 5 vm for maximum compatibility
# see https://github.com/pypa/manylinux
(venv) ubuntu@home:~/galaxy_s4/unicorn/bindings/python$ pip install dist/unicorn-2.0.0-py2.py3-none-linux_aarch64.whl
Processing ./dist/unicorn-2.0.0-py2.py3-none-linux_aarch64.whl
Installing collected packages: unicorn
Successfully installed unicorn-2.0.0
;)
Unicorn is a dependency for a tool I pack inside a Docker container for arm. Just wondering if there is a plan/ETA to add the arm wheel for it? Ty!
Actually this problem also exist on Raspbian arm64:
Building wheel for unicorn (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kob6vese/unicorn_b722d305ecd3466b9831ac9817a04047/setup.py'"'"'; __file__='"'"'/tmp/pip-install-kob6vese/unicorn_b722d305ecd3466b9831ac9817a04047/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-mhhe3zbs
cwd: /tmp/pip-install-kob6vese/unicorn_b722d305ecd3466b9831ac9817a04047/
Complete output (4 lines):
running bdist_wheel
running build
Building C extensions
error: [Errno 2] No such file or directory: '/tmp/pip-install-kob6vese/unicorn_b722d305ecd3466b9831ac9817a04047/../../include/unicorn'
----------------------------------------
ERROR: Failed building wheel for unicorn
You are almost on the right track. The error is that you should use an old enough Linux distribution to build the wheel. In most cases, you could find it here: https://github.com/pypa/manylinux
Or go through another not so backwards compatible track:
diff --git a/bindings/python/build_wheel.sh b/bindings/python/build_wheel.sh index e3c9d444..5429685d 100755 --- a/bindings/python/build_wheel.sh +++ b/bindings/python/build_wheel.sh @@ -10,5 +10,5 @@ else python3 setup.py bdist_wheel fi cd dist -auditwheel repair *.whl -mv -f wheelhouse/*.whl . +#auditwheel repair *.whl +#mv -f wheelhouse/*.whl . diff --git a/bindings/python/setup.py b/bindings/python/setup.py index 51dfb9fe..20f1f4bd 100755 --- a/bindings/python/setup.py +++ b/bindings/python/setup.py @@ -177,7 +177,7 @@ if 'bdist_wheel' in sys.argv and '--plat-name' not in sys.argv: idx = sys.argv.index('bdist_wheel') + 1 sys.argv.insert(idx, '--plat-name') name = get_platform() - if 'linux' in name: + if 'boo' in name: # linux_* platform tags are disallowed because the python ecosystem is fubar # linux builds should be built in the centos 5 vm for maximum compatibility # see https://github.com/pypa/manylinux(venv) ubuntu@home:~/galaxy_s4/unicorn/bindings/python$ pip install dist/unicorn-2.0.0-py2.py3-none-linux_aarch64.whl Processing ./dist/unicorn-2.0.0-py2.py3-none-linux_aarch64.whl Installing collected packages: unicorn Successfully installed unicorn-2.0.0;) Hello. I'm new and I'm trying to install Unicorn and I have an ARM64 on Ubuntu 22. I'm getting the exact same error message as above. For the 24 lines of code you mentioned to switch to aarch64, is this a Python (.py) file that I have to run?
This problem is not specific to Kali. The Python bindings source distribution uploaded to PyPI is broken/unbuildable.
The issue seems to be that setup.py is designed to build from inside the full unicorn repo: https://github.com/unicorn-engine/unicorn/blob/6c1cbef6ac505d355033aef1176b684d02e1eb3a/bindings/python/setup.py#L29 https://github.com/unicorn-engine/unicorn/blob/6c1cbef6ac505d355033aef1176b684d02e1eb3a/bindings/python/setup.py#L96
but the source distribution uploaded to PyPI has a different directory structure where the Unicorn source is placed in a src/ subdirectory: https://github.com/unicorn-engine/unicorn/blob/6c1cbef6ac505d355033aef1176b684d02e1eb3a/bindings/python/setup.py#L48-L81
The problematic UC_DIR variable was introduced in this massive Unicorn 2 commit: aaaea14214ed46ac60cf7ef5766374f660b05777. This change appears to delete the code that supports setup.py being run from both inside the unicorn repo and from the Python source distribution: https://github.com/unicorn-engine/unicorn/blob/c03f50f76a683488afc633cb31be6b9253d37ebf/bindings/python/setup.py#L30
Not sure what the background behind this change was, since I can't find any individual commits from the development of Unicorn 2 - perhaps @aquynh might know?
This problem is not specific to Kali. The Python bindings source distribution uploaded to PyPI is broken/unbuildable.
The issue seems to be that setup.py is designed to build from inside the full unicorn repo:
https://github.com/unicorn-engine/unicorn/blob/6c1cbef6ac505d355033aef1176b684d02e1eb3a/bindings/python/setup.py#L29
https://github.com/unicorn-engine/unicorn/blob/6c1cbef6ac505d355033aef1176b684d02e1eb3a/bindings/python/setup.py#L96
but the source distribution uploaded to PyPI has a different directory structure where the Unicorn source is placed in a src/ subdirectory:
https://github.com/unicorn-engine/unicorn/blob/6c1cbef6ac505d355033aef1176b684d02e1eb3a/bindings/python/setup.py#L48-L81
The problematic UC_DIR variable was introduced in this massive Unicorn 2 commit: aaaea14. This change appears to delete the code that supports setup.py being run from both inside the unicorn repo and from the Python source distribution:
https://github.com/unicorn-engine/unicorn/blob/c03f50f76a683488afc633cb31be6b9253d37ebf/bindings/python/setup.py#L30
Not sure what the background behind this change was, since I can't find any individual commits from the development of Unicorn 2 - perhaps @aquynh might know?
Very good catch, on fixing it.
Fixed in e76b2db434382f59661471faae02b022a3ee5a30. For the wheel, I will open another issue for it as it's mostly CI stuff.
You are almost on the right track. The error is that you should use an old enough Linux distribution to build the wheel. In most cases, you could find it here: https://github.com/pypa/manylinux
Or go through another not so backwards compatible track:
diff --git a/bindings/python/build_wheel.sh b/bindings/python/build_wheel.sh index e3c9d444..5429685d 100755 --- a/bindings/python/build_wheel.sh +++ b/bindings/python/build_wheel.sh @@ -10,5 +10,5 @@ else python3 setup.py bdist_wheel fi cd dist -auditwheel repair *.whl -mv -f wheelhouse/*.whl . +#auditwheel repair *.whl +#mv -f wheelhouse/*.whl . diff --git a/bindings/python/setup.py b/bindings/python/setup.py index 51dfb9fe..20f1f4bd 100755 --- a/bindings/python/setup.py +++ b/bindings/python/setup.py @@ -177,7 +177,7 @@ if 'bdist_wheel' in sys.argv and '--plat-name' not in sys.argv: idx = sys.argv.index('bdist_wheel') + 1 sys.argv.insert(idx, '--plat-name') name = get_platform() - if 'linux' in name: + if 'boo' in name: # linux_* platform tags are disallowed because the python ecosystem is fubar # linux builds should be built in the centos 5 vm for maximum compatibility # see https://github.com/pypa/manylinux(venv) ubuntu@home:~/galaxy_s4/unicorn/bindings/python$ pip install dist/unicorn-2.0.0-py2.py3-none-linux_aarch64.whl Processing ./dist/unicorn-2.0.0-py2.py3-none-linux_aarch64.whl Installing collected packages: unicorn Successfully installed unicorn-2.0.0;)
This also works on Raspbian.