Python from source on armv7l raises ' undefined symbol: __atomic_fetch_add_8 '

[FrancescoScandiffio]

Device	Raspberry Pi 3B revision 2
Architecture	armv7l
OS	Raspberry OS Lite (32 bit)
Python	3.9.2
GCC	10.2.1
Error	ImportError at _sentencepiece.cpython-39-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8

Hello,

I would like to use the sentencepiece python package on a Raspberry Pi 3B with 32 bit Raspi OS but I am experiencing some difficulties with the import. The available wheels are not compatible with my setup so I followed your guide to build the package from source (at https://github.com/google/sentencepiece/blob/master/python/README.md). The installation ends successfully, but when I try to import it raises an ImportError:

Python 3.9.2 (default, Mar 12 2021, 04:06:34) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sentencepiece
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pi/.local/lib/python3.9/site-packages/sentencepiece/__init__.py", line 13, in <module>
    from . import _sentencepiece
ImportError: /home/pi/.local/lib/python3.9/site-packages/sentencepiece/_sentencepiece.cpython-39-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8

Can you help me with this issue? Should I manually link some library or add specific flags to the cmake?

Thank you in advance for any assistance you can provide.

---

Log, from the end of make install

 Install the project...
-- Install configuration: ""
-- Installing: /home/pi/environments/sentencepiece/build/root/lib/pkgconfig/sentencepiece.pc
-- Installing: /home/pi/environments/sentencepiece/build/root/lib/libsentencepiece.a
-- Installing: /home/pi/environments/sentencepiece/build/root/lib/libsentencepiece_train.a
-- Installing: /home/pi/environments/sentencepiece/build/root/bin/spm_encode
-- Installing: /home/pi/environments/sentencepiece/build/root/bin/spm_decode
-- Installing: /home/pi/environments/sentencepiece/build/root/bin/spm_normalize
-- Installing: /home/pi/environments/sentencepiece/build/root/bin/spm_train
-- Installing: /home/pi/environments/sentencepiece/build/root/bin/spm_export_vocab
-- Installing: /home/pi/environments/sentencepiece/build/root/include/sentencepiece_trainer.h
-- Installing: /home/pi/environments/sentencepiece/build/root/include/sentencepiece_processor.h

/usr/lib/python3/dist-packages/setuptools/dist.py:461: UserWarning: Normalizing '0.2.00' to '0.2.0'
  warnings.warn(tmpl.format(**locals()))
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-armv7l-3.9
creating build/lib.linux-armv7l-3.9/sentencepiece
copying src/sentencepiece/__init__.py -> build/lib.linux-armv7l-3.9/sentencepiece
copying src/sentencepiece/_version.py -> build/lib.linux-armv7l-3.9/sentencepiece
copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.linux-armv7l-3.9/sentencepiece
copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.linux-armv7l-3.9/sentencepiece
running build_ext
## cflags=-std=c++17 -I../build/root/include -Wl,-strip-all
## libs=../build/root/lib/libsentencepiece.a ../build/root/lib/libsentencepiece_train.a -Wl,-strip-all
building 'sentencepiece._sentencepiece' extension
creating build/temp.linux-armv7l-3.9
creating build/temp.linux-armv7l-3.9/src
creating build/temp.linux-armv7l-3.9/src/sentencepiece
arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.9 -c src/sentencepiece/sentencepiece_wrap.cxx -o build/temp.linux-armv7l-3.9/src/sentencepiece/sentencepiece_wrap.o -std=c++17 -I../build/root/include -Wl,-strip-all
arm-linux-gnueabihf-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-z,relro -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-armv7l-3.9/src/sentencepiece/sentencepiece_wrap.o -o build/lib.linux-armv7l-3.9/sentencepiece/_sentencepiece.cpython-39-arm-linux-gnueabihf.so ../build/root/lib/libsentencepiece.a ../build/root/lib/libsentencepiece_train.a -Wl,-strip-all
installing to build/bdist.linux-armv7l/wheel
running install
running install_lib
creating build/bdist.linux-armv7l
creating build/bdist.linux-armv7l/wheel
creating build/bdist.linux-armv7l/wheel/sentencepiece
copying build/lib.linux-armv7l-3.9/sentencepiece/__init__.py -> build/bdist.linux-armv7l/wheel/sentencepiece
copying build/lib.linux-armv7l-3.9/sentencepiece/_version.py -> build/bdist.linux-armv7l/wheel/sentencepiece
copying build/lib.linux-armv7l-3.9/sentencepiece/sentencepiece_model_pb2.py -> build/bdist.linux-armv7l/wheel/sentencepiece
copying build/lib.linux-armv7l-3.9/sentencepiece/_sentencepiece.cpython-39-arm-linux-gnueabihf.so -> build/bdist.linux-armv7l/wheel/sentencepiece
copying build/lib.linux-armv7l-3.9/sentencepiece/sentencepiece_pb2.py -> build/bdist.linux-armv7l/wheel/sentencepiece
running install_egg_info
running egg_info
creating src/sentencepiece.egg-info
writing src/sentencepiece.egg-info/PKG-INFO
writing dependency_links to src/sentencepiece.egg-info/dependency_links.txt
writing top-level names to src/sentencepiece.egg-info/top_level.txt
writing manifest file 'src/sentencepiece.egg-info/SOURCES.txt'
reading manifest file 'src/sentencepiece.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'sentencepiece'
warning: no files found matching 'VERSION.*'
writing manifest file 'src/sentencepiece.egg-info/SOURCES.txt'
Copying src/sentencepiece.egg-info to build/bdist.linux-armv7l/wheel/sentencepiece-0.2.0.egg-info
running install_scripts
creating build/bdist.linux-armv7l/wheel/sentencepiece-0.2.0.dist-info/WHEEL
creating 'dist/sentencepiece-0.2.0-cp39-cp39-linux_armv7l.whl' and adding 'build/bdist.linux-armv7l/wheel' to it
adding 'sentencepiece/__init__.py'
adding 'sentencepiece/_sentencepiece.cpython-39-arm-linux-gnueabihf.so'
adding 'sentencepiece/_version.py'
adding 'sentencepiece/sentencepiece_model_pb2.py'
adding 'sentencepiece/sentencepiece_pb2.py'
adding 'sentencepiece-0.2.0.dist-info/METADATA'
adding 'sentencepiece-0.2.0.dist-info/WHEEL'
adding 'sentencepiece-0.2.0.dist-info/top_level.txt'
adding 'sentencepiece-0.2.0.dist-info/RECORD'
removing build/bdist.linux-armv7l/wheel

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing ./dist/sentencepiece-0.2.0-cp39-cp39-linux_armv7l.whl
Installing collected packages: sentencepiece
Successfully installed sentencepiece-0.2.0

[brandonwheat]

Did you ever find a solution to this?

[FrancescoScandiffio]

Did you ever find a solution to this?

Hello @brandonwheat, no I never found any solution. Actually, I didn't even look for it because it was just a feasibility study for a hobby project.

[taku910]

I'm closing this issue now due to a lack of further discussion. Please feel free to reopen it if needed.

[rgommers]

The error indicates a failure to link libatomic. The build config actually does try to do this:

https://github.com/google/sentencepiece/blob/8e05de72a004a513abf62b264cd45eb4b7c3d45a/src/CMakeLists.txt#L209-L221

So if someone else hits this, checking the build output for Found atomic: and adjusting the check linked above should help determine what the fix is.