rerun
rerun copied to clipboard
rerun-io
C++ package manager request tracking issue
For the moment we decided to not support additional build systems and package management for C++, since there's simply too many to choose from and this becomes a bit of a maintenance burden that we're not ready for yet.
This ticket collects the various requests that came up so far and what state they're in:
- Bazel
- There's a minimal example integration here https://github.com/kyle-figure/bazel-minimal-rerun/
- CocoaPods #4568
- Conan
- #4420
- https://github.com/conan-io/conan-center-index/pull/28484
- Conda
- There's a conda forge recipe maintained by @traversaro https://anaconda.org/conda-forge/librerun-sdk
- Gradle #4336
- vcpkg
- https://github.com/microsoft/vcpkg/tree/master/ports/rerun-sdk
@abey79 , @Wumpf I've started work on conan recipe (#4420 ) in our local conan system and faced a few things. Maybe, we could discuss it here or in discord or have a call. Here are the topics:
C++ SDK provided in assets has good and clear process of integration with just couple lines of code in CMake, but it seems to be very important that own code is compiled along with some Rerun sources (*.c - files). However, the classical package should contain only pre-built libraries (lib or dll) and header files. I tried to create conan package and it contains only hpp files with these libs:
- arrow_bundled_dependencies.lib
- arrow_static.lib
- rerun_c__win_x64.lib (from official SDK asset)
- rerun_sdk.lib + rerun_sdk.pdb
Next, when I try to link rerun conan package to existing project XXXX I'm getting these linker errors - apparently libs don't contain all code:
[build] rerun_c__win_x64.lib(re_sdk_comms-0765e134c12468ee.re_sdk_comms.f78a4b73346f03c4-cgu.05.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(re_sdk-9a5f3c7fa3d85f9c.re_sdk.a3283a595e0d2873-cgu.00.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(re_sdk-9a5f3c7fa3d85f9c.re_sdk.a3283a595e0d2873-cgu.12.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(re_sdk_comms-0765e134c12468ee.re_sdk_comms.f78a4b73346f03c4-cgu.01.rcgu.o) : error LNK2001: unresolved external symbol __imp_closesocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSAStartup [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSACleanup [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_freeaddrinfo [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_shutdown [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSAGetLastError [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_setsockopt [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getsockopt [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_recv [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSARecv [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSASend [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_send [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_accept [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_ioctlsocket [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getpeername [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSADuplicateSocketW [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_WSASocketW [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getaddrinfo [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_NtCreateFile [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_RtlNtStatusToDosError [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_NtReadFile [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_NtWriteFile [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_connect [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_select [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_recvfrom [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_BCryptGenRandom [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(getrandom-b857062dfe0c4b33.getrandom.2ae02711c9719d53-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_BCryptGenRandom [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_bind [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_listen [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_getsockname [XXXXXXXXXX.vcxproj]
[build] rerun_c__win_x64.lib(std-4ee9ee8805e6ac55.std.ddad90bab7781587-cgu.0.rcgu.o) : error LNK2001: unresolved external symbol __imp_sendto [XXXXXXXXXX.vcxproj]
[build] XXXXX.exe : fatal error LNK1120: 30 unresolved externals [XXXXXXXXXX.vcxproj]
I would like to understand better the structure of C++ code, how to make it compilable into self-sufficient library and how did you get rerun_c__win_x64.lib - what does it contain in practice.
I'm thinking of maybe forcing linker to produce all symbols by adding following options (not sure it will resolve all linker problems though):
MSVC: /WHOLEARCHIVE option in the linker.
GCC use the -Wl,--whole-archive linker option.
What do you think?
@rgolovanov the missing symbols you hit there are from a bunch of very common system dependencies. These to be accurate ws2_32.dll Bcrypt.dll Userenv.dll ntdll.dll
Afaik whole-archive linking is generally discouraged since it can lead to a lot of bloat (everyone else now using anything from the windows system comes with their own winsock & ntdll etc.). I'm not familiar with Conan, but I'd assume that these system libraries would be explicitly specified as a dependency whenever linking with the static version of Rerun.
@Wumpf thanks for comment. Yes, now I see it should come from here but apparently was cutoff during conan package creation. I'll check what can be done 👍
For what regards C++ SDK support for conda-forge, this is now solved (at least on Linux/macOS) with the librerun-sdk package, see https://github.com/conda-forge/staged-recipes/pull/25648 and https://github.com/conda-forge/librerun-sdk-feedstock .
@traversaro awesome! Would you mind creating a PR that adds this to the docs somewhere in here https://github.com/rerun-io/rerun/tree/main/rerun_cpp#build--distribution ?
@traversaro awesome! Would you mind creating a PR that adds this to the docs somewhere in here https://github.com/rerun-io/rerun/tree/main/rerun_cpp#build--distribution ?
Sure, there are a bit of aspects that I would to tackle before (support for it in the opencv/eigen example, Windows support, general dogfooding it to understand if it works fine), and then I think we can can add it there, does it make sense?
Of course, whenever you feel comfortable with it :)
Now that we have the Windows package as well, I think it is fine to add it: https://github.com/rerun-io/rerun/pull/6381 .
It seems that rerun-sdk was added to vcpkg, related PRs/issues:
- https://github.com/microsoft/vcpkg/pull/44854
- https://github.com/microsoft/vcpkg/pull/45514
- https://github.com/rerun-io/rerun/issues/10033
- https://github.com/rerun-io/rerun/pull/9550
Hi,
I started an attempt to add rerun to conan center: https://github.com/conan-io/conan-center-index/pull/28298
Feel free to look into it and let me know if you have suggestions.
I already have two small questions:
- there are two zip-files in the github-release
rerun_cpp_sdk.zipandrerun_cpp_sdk-0.24.1-multiplatform.zipis there a difference between these - and if yes, which one should be used? - the recipe will generate a target called
rerun_sdk::rerun_sdk- this differs from the target in the zip file, which is calledrerun_sdk. I like the former, because CMake will complain if the target doesn't exist (I think it only does this for targets containing colons) - but I think it would be possible to also make conan generate a target with the current name. Maybe it would be even a good idea to add an alias target to the rerun-cmake?
Thanks and best regards oz
- Maybe it would be even a good idea to add an alias target to the rerun-cmake?
It I am not wrong, the current rerun CMake generates a rerun_sdkTargets.cmake without any namespace, as NAMESPACE is not passed to https://github.com/rerun-io/rerun/blob/4cfb85f0c3e96f36eac3ff692879fddcceb1abc8/rerun_cpp/CMakeLists.txt#L214 . So if we want to start using rerun_sdk::rerun_sdk we need to both add an alias after the add_library (to support FetchContent users) and add a NAMESPACE to install(EXPORT (to support find_package users, including most package managers). Furthermore, we may want to also add ALIAS from rerun_sdk::rerun_sdk to rerun_sdk to avoid a breaking However, I am not familiar with conan, but it is not clear to me where the rerun_sdk::rerun_sdk target that you see is coming from. Is conan changing somehow the rerun_sdkTargets.cmake after it is generated by CMake?
By the way, adding the namespace to CMake targets may be useful for ensuring that everything works fine also with CPS in the future, see https://www.kitware.com/psa-your-package-name-and-target-namespace-should-match/ .
Some possibly-relevant issues and PRs for vcpkg:
- #11269
- https://github.com/microsoft/vcpkg/pull/47404
