xmake icon indicating copy to clipboard operation
xmake copied to clipboard

尝试使用pybind11做python的绑定,按照文档操作无果

Open KaivnD opened this issue 2 years ago • 19 comments

Xmake 版本

v2.6.3+202201242206

操作系统版本和架构

macOS Catalina 10.15.7 (19H15)

描述问题

根据文档尝试使用pybind11

add_rules("mode.release", "mode.debug")
add_requires("pybind11")

target("example")
     add_rules("python.library")
     add_files("src/*.cpp")
     add_packages("pybind11")
     set_languages("c++11")

执行 xmake -vD

[ 25%]: compiling.release src/example.cpp
"/usr/bin/xcrun -sdk macosx clang" -c -Qunused-arguments -arch x86_64 -mmacosx-version-min=10.15 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -O3 -std=c++11 -isystem /Users/kaivnd/.xmake/packages/p/pybind11/v2.9.1/a430ad80c311429aaa6c9bb9e36186a5/include -isystem /Users/kaivnd/.xmake/packages/p/python/3.9.10/a1d024625fd7429983211332bf61ed18/include/python3.9 -isystem /Users/kaivnd/.xmake/packages/o/openssl/1.1.1m/e2d70e9cf7bc4473bf897ffef4cbf761/include -DNDEBUG -o build/.objs/example/macosx/x86_64/release/src/example.cpp.o src/example.cpp
[ 50%]: linking.release _example.dylib
"/usr/bin/xcrun -sdk macosx clang++" -o build/macosx/x86_64/release/_example.dylib build/.objs/example/macosx/x86_64/release/src/example.cpp.o -shared -fPIC -arch x86_64 -mmacosx-version-min=10.15 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -stdlib=libc++ -L/Users/kaivnd/.xmake/packages/p/python/3.9.10/a1d024625fd7429983211332bf61ed18/lib -L/Users/kaivnd/.xmake/packages/o/openssl/1.1.1m/e2d70e9cf7bc4473bf897ffef4cbf761/lib -Wl,-x -lpython3.9 -lssl -lcrypto -lz -install_name @rpath/_example.dylib
[100%]: build ok!

得到的结果是一个dylib,尝试使用python导入这个模块

Python 3.9.7 (default, Sep 16 2021, 08:50:36) 
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import _example
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named '_example'

查阅pybind11文档得知此命令行

c++ -O3 -Wall -shared -std=c++17 -fPIC -undefined dynamic_lookup $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix)

得到文件example.cpython-39-darwin.so,故猜测是后缀名的问题,随后将_example.dylib重命名为_example.cpython-39-darwin.so,再次尝试得到如下错误

Python 3.9.7 (default, Sep 16 2021, 08:50:36) 
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import _example
zsh: segmentation fault  python3

然后我就不知道怎么整了,麻烦大神帮忙看一下!万分感谢

期待的结果

希望可以通过xmake使用pybind11进行C++python的绑定可以正常跨平台使用

工程配置

-- xmake.lua

add_rules("mode.release", "mode.debug")
add_requires("pybind11")

target("example")
     add_rules("python.library")
     add_files("src/*.cpp")
     add_packages("pybind11")
     set_languages("c++11")

// example.cpp
#include <pybind11/pybind11.h>

namespace py = pybind11;

int add(int i, int j) {
    return i + j;
}

PYBIND11_MODULE(_example, m) {
    m.doc() = "pybind11 example plugin"; // optional module docstring

    m.def("add", &add, "A function which adds two numbers");
}

附加信息和错误日志

[ 25%]: compiling.release src/example.cpp
"/usr/bin/xcrun -sdk macosx clang" -c -Qunused-arguments -arch x86_64 -mmacosx-version-min=10.15 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -O3 -std=c++11 -isystem /Users/kaivnd/.xmake/packages/p/pybind11/v2.9.1/a430ad80c311429aaa6c9bb9e36186a5/include -isystem /Users/kaivnd/.xmake/packages/p/python/3.9.10/a1d024625fd7429983211332bf61ed18/include/python3.9 -isystem /Users/kaivnd/.xmake/packages/o/openssl/1.1.1m/e2d70e9cf7bc4473bf897ffef4cbf761/include -DNDEBUG -o build/.objs/example/macosx/x86_64/release/src/example.cpp.o src/example.cpp
[ 50%]: linking.release _example.dylib
"/usr/bin/xcrun -sdk macosx clang++" -o build/macosx/x86_64/release/_example.dylib build/.objs/example/macosx/x86_64/release/src/example.cpp.o -shared -fPIC -arch x86_64 -mmacosx-version-min=10.15 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -stdlib=libc++ -L/Users/kaivnd/.xmake/packages/p/python/3.9.10/a1d024625fd7429983211332bf61ed18/lib -L/Users/kaivnd/.xmake/packages/o/openssl/1.1.1m/e2d70e9cf7bc4473bf897ffef4cbf761/lib -Wl,-x -lpython3.9 -lssl -lcrypto -lz -install_name @rpath/_example.dylib
[100%]: build ok!

KaivnD avatar Mar 18 '22 16:03 KaivnD

用 add_rules("python.library", {soabi = true}) 也不行吗

xq114 avatar Mar 18 '22 16:03 xq114

是的呢,也不行

Python 3.8.2 (default, Dec 21 2020, 15:06:04) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import _example
zsh: segmentation fault  python3

虽然后缀名从dylib变成so了,同样的表现,而且还有另一个问题,with soabi 没有使用conda里的py39,而使用了系统的py38,也挺迷

KaivnD avatar Mar 18 '22 16:03 KaivnD

import _example

看了下,pybind 模块前缀应该是 example.xxx.so 没有下划线,否则 module 导入后报 PyInit__example not found, 我刚改进了下,应该可以了

zsh: segmentation fault python3

测试了下,linux 下加载ok,只有 macos 下加载会有这个问题,不过我这系统环境 gdb/lldb 坏了,暂时没环境调试,你可以自己用 gdb/lldb 加载下 看下哪里崩溃,贴过来

with soabi 没有使用conda里的py39,而使用了系统的py38

项目编译,用的 xmake-repo 里面的 python 库,所以建议保持一致,用 xmake-repo 里面的 python 来加载

$ xrepo env -b python shell
> cd build/macosx/x86_64/release/
> python
Python 3.9.10 (main, Mar 20 2022, 20:41:15)
[Clang 12.0.5 (clang-1205.0.22.11)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import example

waruqi avatar Mar 20 '22 14:03 waruqi

@waruqi lldb没太用明白,不过在一顿折腾之后,系统弹出个窗口,显示一些错误信息,不知道有没有用,如下

Process:               Python [1372]
Path:                  /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.8.2 (3.8.2)
Build Info:            python3-73040006000000~129
Code Type:             X86-64 (Native)
Parent Process:        zsh [716]
Responsible:           Terminal [713]
User ID:               501

Date/Time:             2022-03-22 23:21:38.606 +0800
OS Version:            Mac OS X 10.15.7 (19H15)
Report Version:        12
Anonymous UUID:        FE50A71E-8AB6-3656-1780-CBDCBAC2DE6B


Time Awake Since Boot: 590 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000010
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [1372]

VM Regions Near 0x10:
--> 
    __TEXT                 0000000106273000-0000000106277000 [   16K] r-x/r-x SM=COW  /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   _example.cpython-38-darwin.so 	0x00000001068debc3 0x1067a5000 + 1285059
1   _example.cpython-38-darwin.so 	0x00000001068df3c3 PyEval_RestoreThread + 19 (ceval.c:469)
2   _example.cpython-38-darwin.so 	0x000000010692d108 PyGILState_Ensure + 104
3   _example.cpython-38-darwin.so 	0x00000001067a94fc 0x1067a5000 + 17660
4   _example.cpython-38-darwin.so 	0x00000001067a92fc PyInit__example + 140
5   com.apple.python3             	0x00000001063b3db5 _PyImport_LoadDynamicModuleWithSpec + 613
6   com.apple.python3             	0x00000001063b3660 _imp_create_dynamic + 336
7   com.apple.python3             	0x00000001062e59b1 cfunction_vectorcall_FASTCALL + 177
8   com.apple.python3             	0x00000001062a7124 PyVectorcall_Call + 100
9   com.apple.python3             	0x0000000106383bf7 _PyEval_EvalFrameDefault + 31095
10  com.apple.python3             	0x0000000106388097 _PyEval_EvalCodeWithName + 3287
11  com.apple.python3             	0x00000001062a790d _PyFunction_Vectorcall + 253
12  com.apple.python3             	0x0000000106387012 call_function + 354
13  com.apple.python3             	0x00000001063836d6 _PyEval_EvalFrameDefault + 29782
14  com.apple.python3             	0x00000001062a778d function_code_fastcall + 237
15  com.apple.python3             	0x0000000106387012 call_function + 354
16  com.apple.python3             	0x00000001063836b9 _PyEval_EvalFrameDefault + 29753
17  com.apple.python3             	0x00000001062a778d function_code_fastcall + 237
18  com.apple.python3             	0x0000000106387012 call_function + 354
19  com.apple.python3             	0x000000010638378a _PyEval_EvalFrameDefault + 29962
20  com.apple.python3             	0x00000001062a778d function_code_fastcall + 237
21  com.apple.python3             	0x0000000106387012 call_function + 354
22  com.apple.python3             	0x000000010638378a _PyEval_EvalFrameDefault + 29962
23  com.apple.python3             	0x00000001062a778d function_code_fastcall + 237
24  com.apple.python3             	0x0000000106387012 call_function + 354
25  com.apple.python3             	0x000000010638378a _PyEval_EvalFrameDefault + 29962
26  com.apple.python3             	0x00000001062a778d function_code_fastcall + 237
27  com.apple.python3             	0x00000001062a91bb object_vacall + 459
28  com.apple.python3             	0x00000001062a93f3 _PyObject_CallMethodIdObjArgs + 227
29  com.apple.python3             	0x00000001063b2548 PyImport_ImportModuleLevelObject + 1800
30  com.apple.python3             	0x0000000106381d2d _PyEval_EvalFrameDefault + 23213
31  com.apple.python3             	0x0000000106388097 _PyEval_EvalCodeWithName + 3287
32  com.apple.python3             	0x000000010637c1e0 PyEval_EvalCode + 48
33  com.apple.python3             	0x00000001063cd933 PyRun_FileExFlags + 291
34  com.apple.python3             	0x00000001063ccd9f PyRun_SimpleFileExFlags + 271
35  com.apple.python3             	0x00000001063ec267 Py_RunMain + 2103
36  com.apple.python3             	0x00000001063ec793 pymain_main + 403
37  com.apple.python3             	0x00000001063ec7eb Py_BytesMain + 43
38  libdyld.dylib                 	0x00007fff6b0eccc9 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x0000000106cc5dc0  rcx: 0x0000000000000000  rdx: 0x0000000000000003
  rdi: 0x0000000106cc5dc0  rsi: 0x0000000106a0b386  rbp: 0x00007ffee998ad50  rsp: 0x00007ffee998acf0
   r8: 0x0000000000000039   r9: 0x00007ffee998adc0  r10: 0x0000000106a25ddd  r11: 0x0000000106a25df0
  r12: 0x000000010676cd70  r13: 0x000000010674d8b0  r14: 0x000000010674d620  r15: 0x000000010674d600
  rip: 0x00000001068debc3  rfl: 0x0000000000010246  cr2: 0x0000000000000010
  
Logical CPU:     0
Error Code:      0x00000004 (no mapping for user data read)
Trap Number:     14


Binary Images:
       0x106273000 -        0x106276ff3  com.apple.python3 (3.8.2 - 3.8.2) <B519D60E-3FB8-3BF3-90E1-BF9329E5947F> /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python
       0x106283000 -        0x1064d2ff7  com.apple.python3 (3.8.2 - 3.8.2) <B6FF6B7D-D9D5-3BAF-AEE3-3A8F04133C81> /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Python3
       0x1067a5000 -        0x106a38fff +_example.cpython-38-darwin.so (0) <9CED0B51-52E7-3039-92F1-7025328569A9> /Users/USER/*/_example.cpython-38-darwin.so
       0x106bfb000 -        0x106c8cf47  dyld (750.6) <1D318D60-C9B0-3511-BE9C-82AFD2EF930D> /usr/lib/dyld
       0x106d00000 -        0x106d87ffb +libssl.1.1.dylib (0) <5D583D07-B0B3-3E36-8142-508CA51108A4> /Users/USER/*/libssl.1.1.dylib
       0x106db8000 -        0x10703721b +libcrypto.1.1.dylib (0) <7550BFC6-296C-39A4-8ED7-3BB8C28454D7> /Users/USER/*/libcrypto.1.1.dylib
    0x7fff2407f000 -     0x7fff2408eff7  libSimplifiedChineseConverter.dylib (76) <07231027-C1CD-3E1D-A7A4-F717E91AA91D> /System/Library/CoreServices/Encodings/libSimplifiedChineseConverter.dylib
    0x7fff30fb4000 -     0x7fff31433feb  com.apple.CoreFoundation (6.9 - 1677.104) <C0D70026-EDBE-3CBD-B317-367CF4F1C92F> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
    0x7fff67bfb000 -     0x7fff67bfdff7  libDiagnosticMessagesClient.dylib (112) <C94F3B7B-1854-38EB-9778-834501C53B3F> /usr/lib/libDiagnosticMessagesClient.dylib
    0x7fff680d1000 -     0x7fff680d2fff  libSystem.B.dylib (1281.100.1) <0A6C8BA1-30FD-3D10-83FD-FF29E221AFFE> /usr/lib/libSystem.B.dylib
    0x7fff683b7000 -     0x7fff68409fff  libc++.1.dylib (902.1) <59A8239F-C28A-3B59-B8FA-11340DC85EDC> /usr/lib/libc++.1.dylib
    0x7fff6840a000 -     0x7fff6841fffb  libc++abi.dylib (902) <E692F14F-C65E-303B-9921-BB7E97D77855> /usr/lib/libc++abi.dylib
    0x7fff68e89000 -     0x7fff68e8bfff  libfakelink.dylib (149.1) <36146CB2-E6A5-37BB-9EE8-1B4034D8F3AD> /usr/lib/libfakelink.dylib
    0x7fff68f9b000 -     0x7fff691f2fff  libicucore.A.dylib (64260.0.1) <8AC2CB07-E7E0-340D-A849-186FA1F27251> /usr/lib/libicucore.A.dylib
    0x7fff69f31000 -     0x7fff69f64fde  libobjc.A.dylib (787.1) <6DF81160-5E7F-3E31-AA1E-C875E3B98AF6> /usr/lib/libobjc.A.dylib
    0x7fff6a60d000 -     0x7fff6a61fff3  libz.1.dylib (76) <793D9643-CD83-3AAC-8B96-88D548FAB620> /usr/lib/libz.1.dylib
    0x7fff6aece000 -     0x7fff6aed3ff3  libcache.dylib (83) <AF488D13-9E89-35E0-B078-BE37CC5B8586> /usr/lib/system/libcache.dylib
    0x7fff6aed4000 -     0x7fff6aedffff  libcommonCrypto.dylib (60165.120.1) <C7912BE5-993E-3581-B2A0-6AABDC8C5562> /usr/lib/system/libcommonCrypto.dylib
    0x7fff6aee0000 -     0x7fff6aee7fff  libcompiler_rt.dylib (101.2) <49B8F644-5705-3F16-BBE0-6FFF9B17C36E> /usr/lib/system/libcompiler_rt.dylib
    0x7fff6aee8000 -     0x7fff6aef1ff7  libcopyfile.dylib (166.40.1) <3C481225-21E7-370A-A30E-0CCFDD64A92C> /usr/lib/system/libcopyfile.dylib
    0x7fff6aef2000 -     0x7fff6af84fdb  libcorecrypto.dylib (866.140.1) <60567BF8-80FA-359A-B2F3-A3BAEFB288FD> /usr/lib/system/libcorecrypto.dylib
    0x7fff6b091000 -     0x7fff6b0d1ff0  libdispatch.dylib (1173.100.2) <CD9C059C-91D9-30E8-8926-5B9CD0D5D4F5> /usr/lib/system/libdispatch.dylib
    0x7fff6b0d2000 -     0x7fff6b108fff  libdyld.dylib (750.6) <789A18C2-8AC7-3C88-813D-CD674376585D> /usr/lib/system/libdyld.dylib
    0x7fff6b109000 -     0x7fff6b109ffb  libkeymgr.dylib (30) <DB3337BE-01CA-3425-BD0C-87774FC0CDC0> /usr/lib/system/libkeymgr.dylib
    0x7fff6b117000 -     0x7fff6b117ff7  liblaunch.dylib (1738.140.1) <AFBCBDD3-0B55-3ECD-8E04-A73A3A57356B> /usr/lib/system/liblaunch.dylib
    0x7fff6b118000 -     0x7fff6b11dff7  libmacho.dylib (959.0.1) <AA613A9C-961A-3B67-B696-4622FA59FC4E> /usr/lib/system/libmacho.dylib
    0x7fff6b11e000 -     0x7fff6b120ff3  libquarantine.dylib (110.40.3) <F234E51D-FD0B-3EE4-B679-AE3EE9C536C3> /usr/lib/system/libquarantine.dylib
    0x7fff6b121000 -     0x7fff6b122ff7  libremovefile.dylib (48) <7C7EFC79-BD24-33EF-B073-06AED234593E> /usr/lib/system/libremovefile.dylib
    0x7fff6b123000 -     0x7fff6b13aff3  libsystem_asl.dylib (377.60.2) <1563EE02-0657-3B78-99BE-A947C24122EF> /usr/lib/system/libsystem_asl.dylib
    0x7fff6b13b000 -     0x7fff6b13bff7  libsystem_blocks.dylib (74) <0D53847E-AF5F-3ACF-B51F-A15DEA4DEC58> /usr/lib/system/libsystem_blocks.dylib
    0x7fff6b13c000 -     0x7fff6b1c3fff  libsystem_c.dylib (1353.100.2) <BBDED5E6-A646-3EED-B33A-91E4331EA063> /usr/lib/system/libsystem_c.dylib
    0x7fff6b1c4000 -     0x7fff6b1c7ffb  libsystem_configuration.dylib (1061.141.1) <0EE84C33-64FD-372B-974A-AF7A136F2068> /usr/lib/system/libsystem_configuration.dylib
    0x7fff6b1c8000 -     0x7fff6b1cbfff  libsystem_coreservices.dylib (114) <A199156E-058D-3ABB-BCE9-4B9F20DCED0F> /usr/lib/system/libsystem_coreservices.dylib
    0x7fff6b1cc000 -     0x7fff6b1d4fff  libsystem_darwin.dylib (1353.100.2) <5B12B5DB-3F30-37C1-8ECC-49A66B1F2864> /usr/lib/system/libsystem_darwin.dylib
    0x7fff6b1d5000 -     0x7fff6b1dcfff  libsystem_dnssd.dylib (1096.100.3) <EBB4C2C2-E031-3094-B40A-E67BF261D295> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff6b1dd000 -     0x7fff6b1deffb  libsystem_featureflags.dylib (17) <29FD922A-EC2C-3F25-BCCC-B58D716E60EC> /usr/lib/system/libsystem_featureflags.dylib
    0x7fff6b1df000 -     0x7fff6b22cff7  libsystem_info.dylib (538) <8A321605-5480-330B-AF9E-64E65DE61747> /usr/lib/system/libsystem_info.dylib
    0x7fff6b22d000 -     0x7fff6b259ff7  libsystem_kernel.dylib (6153.141.2.2) <5CDBBC06-6CA6-3432-9FDA-681047866F3E> /usr/lib/system/libsystem_kernel.dylib
    0x7fff6b25a000 -     0x7fff6b2a1fff  libsystem_m.dylib (3178) <00F331F1-0D09-39B3-8736-1FE90E64E903> /usr/lib/system/libsystem_m.dylib
    0x7fff6b2a2000 -     0x7fff6b2c9fff  libsystem_malloc.dylib (283.100.6) <8549294E-4C53-36EB-99F3-584A7393D8D5> /usr/lib/system/libsystem_malloc.dylib
    0x7fff6b2ca000 -     0x7fff6b2d7ffb  libsystem_networkextension.dylib (1095.140.2) <F06C65C5-2CBE-313C-96E1-A09240F9FE57> /usr/lib/system/libsystem_networkextension.dylib
    0x7fff6b2d8000 -     0x7fff6b2e1ff7  libsystem_notify.dylib (241.100.2) <FA22F928-D91B-3AA5-96BB-3186AC0FB264> /usr/lib/system/libsystem_notify.dylib
    0x7fff6b2e2000 -     0x7fff6b2eafef  libsystem_platform.dylib (220.100.1) <009A7C1F-313A-318E-B9F2-30F4C06FEA5C> /usr/lib/system/libsystem_platform.dylib
    0x7fff6b2eb000 -     0x7fff6b2f5fff  libsystem_pthread.dylib (416.100.3) <62CB1A98-0B8F-31E7-A02B-A1139927F61D> /usr/lib/system/libsystem_pthread.dylib
    0x7fff6b2f6000 -     0x7fff6b2faff3  libsystem_sandbox.dylib (1217.141.2) <051C4018-4345-3034-AC98-6DE42FB8273B> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff6b2fb000 -     0x7fff6b2fdfff  libsystem_secinit.dylib (62.100.2) <F80872AA-E1FD-3D7E-8729-467656EC6561> /usr/lib/system/libsystem_secinit.dylib
    0x7fff6b2fe000 -     0x7fff6b305ffb  libsystem_symptoms.dylib (1238.120.1) <5820A2AF-CE72-3AB3-ABCC-273A3419FB55> /usr/lib/system/libsystem_symptoms.dylib
    0x7fff6b306000 -     0x7fff6b31cff2  libsystem_trace.dylib (1147.120) <04B47629-847B-3D74-8ABE-C05EF9DEEFE4> /usr/lib/system/libsystem_trace.dylib
    0x7fff6b31e000 -     0x7fff6b323ff7  libunwind.dylib (35.4) <42B7B509-BAFE-365B-893A-72414C92F5BF> /usr/lib/system/libunwind.dylib
    0x7fff6b324000 -     0x7fff6b359ffe  libxpc.dylib (1738.140.1) <3E243A41-030F-38E3-9FD2-7B38C66C35B1> /usr/lib/system/libxpc.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 6246
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=410.5M resident=0K(0%) swapped_out_or_unallocated=410.5M(100%)
Writable regions: Total=46.1M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=46.1M(100%)
 
                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        1 
MALLOC                            27.5M       15 
MALLOC guard page                   16K        4 
MALLOC_LARGE (reserved)            640K        1         reserved VM address space (unallocated)
STACK GUARD                          4K        1 
Stack                             16.0M        1 
VM_ALLOCATE                       1536K        6 
__DATA                            2840K       54 
__DATA_CONST                       260K        4 
__LINKEDIT                       391.0M       11 
__OBJC_RO                         32.3M        1 
__OBJC_RW                         1908K        2 
__TEXT                            19.6M       50 
__UNICODE                          564K        1 
shared memory                       12K        3 
===========                     =======  ======= 
TOTAL                            493.9M      155 
TOTAL, minus reserved VM space   493.3M      155 

Model: MacBookPro14,1, BootROM 428.0.0.0.0, 2 processors, Dual-Core Intel Core i5, 2.3 GHz, 8 GB, SMC 2.43f10
Graphics: kHW_IntelIrisGraphics640Item, Intel Iris Plus Graphics 640, spdisplays_builtin
Memory Module: BANK 0/DIMM0, 4 GB, LPDDR3, 2133 MHz, 0x802C, 0x4D5435324C3531324D3332443250462D3039
Memory Module: BANK 1/DIMM0, 4 GB, LPDDR3, 2133 MHz, 0x802C, 0x4D5435324C3531324D3332443250462D3039
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x170), Broadcom BCM43xx 1.0 (7.77.111.1 AirPortDriverBrcmNIC-1615.2)
Bluetooth: Version 7.0.6f7, 3 services, 27 devices, 1 incoming serial ports
Network Service: Wi-Fi, AirPort, en0
USB Device: USB 3.0 Bus
Thunderbolt Bus: MacBook Pro, Apple Inc., 41.4

KaivnD avatar Mar 22 '22 15:03 KaivnD

看栈是 崩在 python 里面了,感觉跟 xmake 关系不大,要么是 pybind11 问题,要么是 python 的问题,你可以到 pybind11 issues 上去问问。。

或者你可以用 cmake 啥的其他构建工具去构建一个模块,导入下试试,看看是否也会崩,如果 ok,可以对比下 两边的 flags 差异。

我看 pybind11 issues 也有人反馈类似的问题

https://github.com/pybind/pybind11/issues/3054 https://github.com/pybind/pybind11/issues/2558

waruqi avatar Mar 22 '22 15:03 waruqi

https://github.com/pybind/pybind11/issues/1579 这个似乎跟这个类似,上面说是 python 版本冲突问题

waruqi avatar Mar 22 '22 15:03 waruqi

@waruqi 但是使用c++ -O3 -Wall -shared -std=c++17 -fPIC -undefined dynamic_lookup $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix) 指令能行

KaivnD avatar Apr 06 '22 06:04 KaivnD

@waruqi 但是使用c++ -O3 -Wall -shared -std=c++17 -fPIC -undefined dynamic_lookup $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix) 指令能行

对比一下,只有-undefined dynamic_lookup缺了,你试试手动添加这个flag,add_cxxflags("-undefined dynamic_lookup", {force=true})编译试试

xq114 avatar Apr 06 '22 08:04 xq114

还是不行,🤦‍♂️

KaivnD avatar Apr 06 '22 13:04 KaivnD

@waruqi 但是使用c++ -O3 -Wall -shared -std=c++17 -fPIC -undefined dynamic_lookup $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix) 指令能行

既然这个可以,那就跟 xmake -vD 做下对比,复制里面的命令,挨个测试下,哪个 flags 有差异

waruqi avatar Apr 06 '22 13:04 waruqi

还是不行,🤦‍♂️

我看你之前测试用的python版本都不一样,一下3.9.7一下3.8.2的,能不能用xmake安装的python 3.9.10试试?运行xrepo env -b python3 shell就可以激活虚拟环境,在虚拟环境里面运行python3

xq114 avatar Apr 06 '22 14:04 xq114

@xq114 不对,build之后得到的后缀是py38的

KaivnD avatar Apr 06 '22 15:04 KaivnD

@waruqi 这是日志

checkinfo: cannot runv(ccache --version), No such file or directory
checking for ccache ... no
[ 25%]: compiling.release src/example.cpp
"/usr/bin/xcrun -sdk macosx clang" -c -Qunused-arguments -arch x86_64 -mmacosx-version-min=10.15 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -O3 -std=c++11 -isystem /Users/kaivnd/.xmake/packages/p/pybind11/v2.9.1/a430ad80c311429aaa6c9bb9e36186a5/include -isystem /Users/kaivnd/.xmake/packages/p/python/3.9.10/a1d024625fd7429983211332bf61ed18/include/python3.9 -isystem /Users/kaivnd/.xmake/packages/o/openssl/1.1.1m/e2d70e9cf7bc4473bf897ffef4cbf761/include -undefined dynamic_lookup -DNDEBUG -o build/.objs/example/macosx/x86_64/release/src/example.cpp.o src/example.cpp
checking for flags (-MMD -MF) ... ok
> xcrun -sdk macosx clang "-MMD" "-MF" "/dev/null"
checking for flags (-fdiagnostics-color=always) ... ok
> xcrun -sdk macosx clang "-fdiagnostics-color=always"
[ 50%]: linking.release _example.cpython-38-darwin.so
"/usr/bin/xcrun -sdk macosx clang++" -o build/macosx/x86_64/release/_example.cpython-38-darwin.so build/.objs/example/macosx/x86_64/release/src/example.cpp.o -shared -fPIC -arch x86_64 -mmacosx-version-min=10.15 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -stdlib=libc++ -L/Users/kaivnd/.xmake/packages/p/python/3.9.10/a1d024625fd7429983211332bf61ed18/lib -L/Users/kaivnd/.xmake/packages/o/openssl/1.1.1m/e2d70e9cf7bc4473bf897ffef4cbf761/lib -Wl,-x -lpython3.9 -lssl -lcrypto -lz -install_name @rpath/_example.cpython-38-darwin.so
[100%]: build ok!

KaivnD avatar Apr 06 '22 15:04 KaivnD

@xq114 不对,build之后得到的后缀是py38的

在虚拟环境里面xmake clean --all 重新build也是py38吗?那样就是find_python的查找有问题了

xq114 avatar Apr 08 '22 00:04 xq114

https://pybind11.readthedocs.io/en/latest/basics.html#creating-bindings-for-a-simple-function

这里的example没有下划线,我试了下windows上原始版本的代码是ok的 image

xq114 avatar Apr 19 '22 11:04 xq114

抱歉过了那么长时间才跟进,前一段时间比较忙,昨天把Mac升级到 12.5 (21G72)最新版了,这个版本的内置python被去掉了,干净的系统,用conda做虚拟环境,又做了一次实验,问题依然存在。

KaivnD avatar Aug 08 '22 14:08 KaivnD

这个问题 其实我这 mac 下也是一样的问题,但我也不知道为啥。也没时间细看

waruqi avatar Aug 08 '22 14:08 waruqi

https://github.com/pybind/pybind11/issues/3907

image

这里说要加-undefined dynamic_lookup (编译跟链接都要加),而且不能直接链接到python的库

xq114 avatar Aug 09 '22 13:08 xq114

pybind/pybind11#3907 这里说要加-undefined dynamic_lookup (编译跟链接都要加),而且不能直接链接到python的库

确实是这个问题,但跟编译无关,是链接的问题,链接时候加上 -undefined dynamic_lookup ,然后还得去掉 -lpython3.9 的链接

之前 -undefined dynamic_lookup 试过,但是之前没去掉 python 的 links ,所以一直不行。。

我刚改了下,应该可以了,这边测试修复了。。xmake update dev

waruqi avatar Aug 09 '22 14:08 waruqi

@waruqi 亲测有效,给大佬点赞!:smile: 那这个issue需要关掉吗?

KaivnD avatar Aug 11 '22 14:08 KaivnD