pybind11
pybind11 copied to clipboard
[BUG]: relocation R_X86_64_TPOFF32 against hidden symbol `_ZZN8pybind116handle15inc_ref_counterEmE7counter' can not be used when making a shared object
Required prerequisites
- [X] Make sure you've read the documentation. Your issue may be addressed there.
- [X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
- [X] Consider asking first in the Gitter chat room or in a Discussion.
What version (or hash if on master) of pybind11 are you using?
2.10.1
Problem description
/usr/bin/ld: build/temp.linux-x86_64-3.10/csrc/transformer/ds_transformer_cuda.o: relocation R_X86_64_TPOFF32 against hidden symbol `_ZZN8pybind116handle15inc_ref_counterEmE7counter' can not be used when making a shared object
Reproducible example code
I was following https://www.deepspeed.ai/tutorials/advanced-install/.
Specifically, it can be reproduced with
`DS_BUILD_TRANSFORMER_INFERENCE=1 pip install .`
The command that failed is
root@4212f9fa521e:/app# x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 -Wl,-Bsymbolic-functions -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.10/csrc/transformer/cublas_wrappers.o build/temp.linux-x86_64-3.10/csrc/transformer/dropout_kernels.o build/temp.linux-x86_64-3.10/csrc/transformer/ds_transformer_cuda.o build/temp.linux-x86_64-3.10/csrc/transformer/gelu_kernels.o build/temp.linux-x86_64-3.10/csrc/transformer/general_kernels.o build/temp.linux-x86_64-3.10/csrc/transformer/normalize_kernels.o build/temp.linux-x86_64-3.10/csrc/transformer/softmax_kernels.o build/temp.linux-x86_64-3.10/csrc/transformer/transform_kernels.o -L/usr/local/lib/python3.10/dist-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.10/deepspeed/ops/transformer/stochastic_transformer_op.cpython-310-x86_64-linux-gnu.so
### Is this a regression? Put the last known working version here if it is.
Not a regression
Adding '-fPIC' to this line can solve the problem.
https://github.com/microsoft/DeepSpeed/blob/3418b869f0ec784b90a35c1253320780b03645a9/op_builder/builder.py#L670