setup.py Windows linking error

[artkuli]

trafficstars

Hi, I got a problem with building python egg file with --cpp_implementation I've tried to build it with protobuf 3.15.0 on Windows 10 I got an error that the linker cannot find 16 symbols

My steps to reproduce :

1. git clone https://github.com/google/protobuf.git
2. cd protobuf
3. git checkout v3.15.0
4. cd cmake
5. mkdir -p build\solution
6. cmake -G "Visual Studio 16 2019" ../.. --config Release
7. In Visual Studio I set up a properties for libprotobuf and libprotobuf-lite as Runtime Library (Multi-thread DLL (/MD))
8. build targets: libprotobuf libprotobuf-lite libprotoc protoc
9. Add path to the PATH env: set PATH=%PATH%;C:\Path\to\protobuf\cmake\build\solution\Release
10. cd C:\Path\to\protobuf\python
11. Change line in setup.py from: ​libraries = ['protobuf'] to libraries = ['libprotobuf', 'libprotobuf-lite']
12. Change line in setup.py from: extra_objects = ['../src/.libs/libprotobuf.a', '../src/.libs/libprotobuf-lite.a'] to extra_objects = ['../cmake/build/solution/Release/libprotobuf.lib', '../cmake/build/solution/Release/libprotobuf-lite.lib']
13. Change line in setup.py from: library_dirs=['../src/.libs'], to: library_dirs=['../src/.libs', '../cmake/build/solution/Release'],
14. Run: python setup.py build –cpp_implementation

On the last step I got an error:

C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:/Users/akulikow/source/repos/protobuf/cmake/build/solution/Release /LIBPATH:C:/Users/akulikow/cmake /LIBPATH:C:/Users/akulikow/cmake/lib /LIBPATH:C:/Users/akulikow/cmake/bin /LIBPATH:c:\Users\akulikow\python37_proto\libs /LIBPATH:C:\Users\akulikow\AppData\Local\Programs\Python\Python37\libs /LIBPATH:C:\Users\akulikow\AppData\Local\Programs\Python\Python37 /LIBPATH:c:\Users\akulikow\python37_proto\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64" libprotobuf.lib libprotobuf-lite.lib /EXPORT:PyInit__message build\temp.win-amd64-3.7\Release\google/protobuf/pyext\descriptor.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\descriptor_containers.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\descriptor_database.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\descriptor_pool.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\extension_dict.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\field.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\map_container.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\message.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\message_factory.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\message_module.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\repeated_composite_container.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\repeated_scalar_container.obj build\temp.win-amd64-3.7\Release\google/protobuf/pyext\unknown_fields.obj /OUT:build\lib.win-amd64-3.7\google\protobuf\pyext\_message.cp37-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.7\Release\google/protobuf/pyext\_message.cp37-win_amd64.lib
   Creating library build\temp.win-amd64-3.7\Release\google/protobuf/pyext\_message.cp37-win_amd64.lib and object build\temp.win-amd64-3.7\Release\google/protobuf/pyext\_message.cp37-win_amd64.exp
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::ServiceDescriptorProtoDefaultTypeInternal google::protobuf::_ServiceDescriptorProto_default_instance_" (?_ServiceDescriptorProto_default_instance_@protobuf@google@@3UServiceDescriptorProtoDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::DescriptorProtoDefaultTypeInternal google::protobuf::_DescriptorProto_default_instance_" (?_DescriptorProto_default_instance_@protobuf@google@@3UDescriptorProtoDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "private: static int google::protobuf::io::CodedInputStream::default_recursion_limit_" (?default_recursion_limit_@CodedInputStream@io@protobuf@google@@0HA)
descriptor.obj : error LNK2001: unresolved external symbol "private: static enum google::protobuf::FieldDescriptor::CppType const * const google::protobuf::FieldDescriptor::kTypeToCppTypeMap" (?kTypeToCppTypeMap@FieldDescriptor@protobuf@google@@0QBW4CppType@123@B)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::MessageOptionsDefaultTypeInternal google::protobuf::_MessageOptions_default_instance_" (?_MessageOptions_default_instance_@protobuf@google@@3UMessageOptionsDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::OneofOptionsDefaultTypeInternal google::protobuf::_OneofOptions_default_instance_" (?_OneofOptions_default_instance_@protobuf@google@@3UOneofOptionsDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::EnumOptionsDefaultTypeInternal google::protobuf::_EnumOptions_default_instance_" (?_EnumOptions_default_instance_@protobuf@google@@3UEnumOptionsDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::EnumValueOptionsDefaultTypeInternal google::protobuf::_EnumValueOptions_default_instance_" (?_EnumValueOptions_default_instance_@protobuf@google@@3UEnumValueOptionsDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::EnumDescriptorProtoDefaultTypeInternal google::protobuf::_EnumDescriptorProto_default_instance_" (?_EnumDescriptorProto_default_instance_@protobuf@google@@3UEnumDescriptorProtoDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::FileDescriptorProtoDefaultTypeInternal google::protobuf::_FileDescriptorProto_default_instance_" (?_FileDescriptorProto_default_instance_@protobuf@google@@3UFileDescriptorProtoDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::FileOptionsDefaultTypeInternal google::protobuf::_FileOptions_default_instance_" (?_FileOptions_default_instance_@protobuf@google@@3UFileOptionsDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::MethodDescriptorProtoDefaultTypeInternal google::protobuf::_MethodDescriptorProto_default_instance_" (?_MethodDescriptorProto_default_instance_@protobuf@google@@3UMethodDescriptorProtoDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "struct google::protobuf::FieldOptionsDefaultTypeInternal google::protobuf::_FieldOptions_default_instance_" (?_FieldOptions_default_instance_@protobuf@google@@3UFieldOptionsDefaultTypeInternal@12@A)
descriptor.obj : error LNK2001: unresolved external symbol "const google::protobuf::FileDescriptorProto::`vftable'" (??_7FileDescriptorProto@protobuf@google@@6B@)
map_container.obj : error LNK2001: unresolved external symbol "private: static char const * const * const google::protobuf::FieldDescriptor::kCppTypeToName" (?kCppTypeToName@FieldDescriptor@protobuf@google@@0QBQEBDB)
message.obj : error LNK2001: unresolved external symbol "float __cdecl google::protobuf::io::SafeDoubleToFloat(double)" (?SafeDoubleToFloat@io@protobuf@google@@YAMN@Z)
build\lib.win-amd64-3.7\google\protobuf\pyext\_message.cp37-win_amd64.pyd : fatal error LNK1120: 16 unresolved externals
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Professional\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\link.exe' failed with exit status 1120

When I run directly the linker with these parameters in verbose mode I see as a result:

Finished searching libraries
Searching libraries
    Searching C:\Users\akulikow\source\repos\protobuf\cmake\build\solution\Release\libprotobuf.lib:
    Searching C:\Users\akulikow\source\repos\protobuf\cmake\build\solution\Release\libprotobuf-lite.lib:
    Searching C:\Users\akulikow\AppData\Local\Programs\Python\Python37\libs\python37.lib:
    Searching C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\lib\x64\libcpmt.lib:
    Searching C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\lib\x64\LIBCMT.lib:
    Searching C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\lib\x64\OLDNAMES.lib:
    Searching C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64\kernel32.lib:
    Searching C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\lib\x64\libvcruntime.lib:
    Searching C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64\libucrt.lib:
    Searching C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64\uuid.lib:

Finished searching libraries

Finished pass 1

It looks like a searched symbols are not in the libraries. Do you have idea what is going on?

[frankgray]

+1 protobuf-python-4.21.2

protobuf.lib(zero_copy_stream_impl.obj) : error LNK2001: 无法解析的外部符号 __imp_strerror
protobuf.lib(io_win32.obj) : error LNK2001: 无法解析的外部符号 __imp__wfopen
protobuf.lib(io_win32.obj) : error LNK2001: 无法解析的外部符号 __imp__wchdir
protobuf.lib(io_win32.obj) : error LNK2001: 无法解析的外部符号 __imp__wmkdir
protobuf.lib(io_win32.obj) : error LNK2001: 无法解析的外部符号 __imp__waccess
protobuf.lib(io_win32.obj) : error LNK2001: 无法解析的外部符号 __imp__wsopen_dispatch
protobuf.lib(io_win32.obj) : error LNK2001: 无法解析的外部符号 __imp__wstat64i32
protobuf.lib(io_win32.obj) : error LNK2001: 无法解析的外部符号 __imp__dup
protobuf.lib(io_win32.obj) : error LNK2001: 无法解析的外部符号 __imp__dup2
protobuf.lib(io_win32.obj) : error LNK2001: 无法解析的外部符号 __imp__setmode
build\lib.win-amd64-3.10\google\protobuf\pyext\_message.cp310-win_amd64.pyd : fatal error LNK1120: 10 个无法解析的外部命令
error: command 'D:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120

[anandolee]

Checked with other team member,

Copy from chat with Mike "I never tried this on windows but I had a hell of a time fighting similar errors on linux

it's possible https://github.com/protocolbuffers/protobuf/pull/10132 fixes it?"

[ricrogz]

In case someone hits this (I just did), you might want to check if you built protobuf cpp code as shared libs. If so, you should use python setup.py build_ext -DPROTOBUF_USE_DLLS –cpp_implementation, as mentioned in https://github.com/protocolbuffers/protobuf/blob/main/cmake/README.md#dlls-vs-static-linking