sentencepiece icon indicating copy to clipboard operation
sentencepiece copied to clipboard

The pip command to install the SentencePiece Python module fails.

Open tprrt opened this issue 1 year ago • 8 comments

pip install sentencepiece Collecting sentencepiece Using cached sentencepiece-0.2.0.tar.gz (2.6 MB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: sentencepiece Building wheel for sentencepiece (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for sentencepiece (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [108 lines of output] /tmp/pip-build-env-fvng4eb3/overlay/lib/python3.13/site-packages/setuptools/_distutils/dist.py:261: UserWarning: Unknown distribution option: 'test_suite' warnings.warn(msg) running bdist_wheel running build running build_py creating build/lib.linux-x86_64-cpython-313/sentencepiece copying src/sentencepiece/init.py -> build/lib.linux-x86_64-cpython-313/sentencepiece copying src/sentencepiece/_version.py -> build/lib.linux-x86_64-cpython-313/sentencepiece copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.linux-x86_64-cpython-313/sentencepiece copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.linux-x86_64-cpython-313/sentencepiece running build_ext Package sentencepiece was not found in the pkg-config search path. Perhaps you should add the directory containing `sentencepiece.pc' to the PKG_CONFIG_PATH environment variable Package 'sentencepiece' not found ./build_bundled.sh: line 21: cmake: command not found ./build_bundled.sh: line 22: cmake: command not found Traceback (most recent call last): File "/home/tprrt/venv/lib64/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() ~~~~^^ File "/home/tprrt/venv/lib64/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ~~~~^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/tprrt/venv/lib64/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel return _build_backend().build_wheel(wheel_directory, config_settings, ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ metadata_directory) ^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-fvng4eb3/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 438, in build_wheel return _build(['bdist_wheel', '--dist-info-dir', str(metadata_directory)]) File "/tmp/pip-build-env-fvng4eb3/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 426, in _build return self._build_with_temp_dir( ~~~~~~~~~~~~~~~~~~~~~~~~~^ cmd, ^^^^ ...<3 lines>... self._arbitrary_args(config_settings), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/tmp/pip-build-env-fvng4eb3/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 407, in _build_with_temp_dir self.run_setup() ~~~~~~~~~~~~~~^^ File "/tmp/pip-build-env-fvng4eb3/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 522, in run_setup super().run_setup(setup_script=setup_script) ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-fvng4eb3/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 320, in run_setup exec(code, locals()) ~~~~^^^^^^^^^^^^^^^^ File "", line 169, in File "/tmp/pip-build-env-fvng4eb3/overlay/lib/python3.13/site-packages/setuptools/init.py", line 117, in setup return distutils.core.setup(**attrs) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ [...]

tprrt avatar Nov 20 '24 09:11 tprrt

Indeed, same problem for me (and a friend who also tested), both running on Arch Linux.

I suspect this came from the upgrade to Python 3.13, but I'm not sure.

Vectorrent avatar Jan 01 '25 17:01 Vectorrent

Try installing cmake first. As per the following excerpt from your log file, cmake command cannot be found.

./build_bundled.sh: line 21: cmake: command not found
./build_bundled.sh: line 22: cmake: command not found

In MacOS, you can use brew: brew install cmake In Debian-based Linux, you can use apt: sudo apt install cmake

DalianisDim avatar Jan 03 '25 11:01 DalianisDim

Try installing cmake first. As per the following excerpt from your log file, cmake command cannot be found.

This was the trick, thank you! I can install SentencePiece on Arch now.

Vectorrent avatar Jan 03 '25 16:01 Vectorrent

pip install cmake was the solution.

Frontrider avatar Feb 10 '25 22:02 Frontrider

I've installed cmake, both via my distro package manager and via pip in the conda env, and I'm still getting the same error.

hchasens avatar Apr 29 '25 19:04 hchasens

Same here

NXTler avatar May 10 '25 15:05 NXTler

Same here

rcanand avatar May 20 '25 21:05 rcanand

The issue is that this branch is not yet released #1084 so until it's released it won't work on 3.13

imadreamerboy avatar May 22 '25 16:05 imadreamerboy

Closed by #1120

crusaderky avatar Jun 27 '25 13:06 crusaderky

Fixed in https://github.com/google/sentencepiece/releases/tag/v0.2.1

taku910 avatar Aug 12 '25 07:08 taku910