analogdevicesinc
build error -DCPP_BINDINGS=ON and -DWITH_EXTERNAL_BACKEND=ON
This builds fine:
cmake ../ -DBUILD_SHARED_LIBS=ON -DCOMPILE_WARNING_AS_ERROR=ON -DCPP_BINDINGS=OFF -DCSHARP_BINDINGS=OFF -DENABLE_IPV6=OFF -DHAVE_DNS_SD=OFF -DLIBIIO_COMPAT=OFF -DNO_THREADS=OFF -DOSX_FRAMEWORK=OFF -DPYTHON_BINDINGS=OFF -DWITH_AIO=OFF -DWITH_EXAMPLES=OFF -DWITH_EXTERNAL_BACKEND=ON -DWITH_GCOV=OFF -DWITH_HWMON=OFF -DWITH_IIOD=OFF -DWITH_LIBTINYIIOD=OFF -DWITH_LOCAL_BACKEND=OFF -DWITH_LOCAL_CONFIG=OFF -DWITH_LOCAL_DMABUF_API=OFF -DWITH_LOCAL_MMAP_API=OFF -DWITH_MAN=OFF -DWITH_MODULES=OFF -DWITH_NETWORK_BACKEND=OFF -DWITH_NETWORK_BACKEND_DYNAMIC=OFF -DWITH_SERIAL_BACKEND=OFF -DWITH_SERIAL_BACKEND_DYNAMIC=OFF -DWITH_USB_BACKEND=OFF -DWITH_USB_BACKEND_DYNAMIC=OFF -DWITH_UTILS=OFF -DWITH_XML_BACKEND=OFF -DWITH_ZSTD=OFF
-- cmake version: 3.18.4
-- The C compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test HAS_WPEDANTIC
-- Performing Test HAS_WPEDANTIC - Success
-- Performing Test HAS_WSHADOW
-- Performing Test HAS_WSHADOW - Success
-- Looking for strdup
-- Looking for strdup - found
-- Looking for strndup
-- Looking for strndup - found
-- Looking for strerror_r
-- Looking for strerror_r - found
-- Looking for strtok_r
-- Looking for strtok_r - found
-- Looking for newlocale
-- Looking for newlocale - found
-- Setting -Werror
-- Found Git: /usr/bin/git (found version "2.30.2")
-- Looking for pthread_setname_np
-- Looking for pthread_setname_np - found
-- Features enabled : external
-- Features disabled: threadless compat xml zstd network dns-sd avahi bonjour ipv6 serial local local-dmabuf local-mmap hwmon usb utils examples libtinyiiod iiod modules usb-dynamic network-dynamic serial-dynamic udev-rule doc man man-utils python-bindings c#-bindings c++-bindings
-- LOG_LEVEL set to "Info"
-- Configuring done
-- Generating done
-- Build files have been written to: /home/rgetz/github/libiio/tmp.EmHaJTcv92
Scanning dependencies of target iio
[ 23%] Building C object CMakeFiles/iio.dir/context.c.o
[ 23%] Building C object CMakeFiles/iio.dir/channel.c.o
[ 23%] Building C object CMakeFiles/iio.dir/backend.c.o
[ 23%] Building C object CMakeFiles/iio.dir/attr.c.o
[ 29%] Building C object CMakeFiles/iio.dir/device.c.o
[ 35%] Building C object CMakeFiles/iio.dir/events.c.o
[ 41%] Building C object CMakeFiles/iio.dir/block.c.o
[ 47%] Building C object CMakeFiles/iio.dir/buffer.c.o
[ 52%] Building C object CMakeFiles/iio.dir/library.c.o
[ 58%] Building C object CMakeFiles/iio.dir/mask.c.o
[ 64%] Building C object CMakeFiles/iio.dir/scan.c.o
[ 70%] Building C object CMakeFiles/iio.dir/task.c.o
[ 76%] Building C object CMakeFiles/iio.dir/stream.c.o
[ 82%] Building C object CMakeFiles/iio.dir/sort.c.o
[ 88%] Building C object CMakeFiles/iio.dir/utilities.c.o
[ 94%] Building C object CMakeFiles/iio.dir/lock.c.o
[100%] Linking C static library libiio.a
[100%] Built target iio
0ea46d53cb9c9d0cd9aa9a6922d3a631 libiio.so
but adding the c++ bindings causes a build failure:
cmake ../ -DBUILD_SHARED_LIBS=OFF -DCOMPILE_WARNING_AS_ERROR=OFF -DCPP_BINDINGS=ON -DCSHARP_BINDINGS=OFF -DENABLE_IPV6=OFF -DHAVE_DNS_SD=OFF -DLIBIIO_COMPAT=OFF -DNO_THREADS=OFF -DOSX_FRAMEWORK=OFF -DPYTHON_BINDINGS=OFF -DWITH_AIO=OFF -DWITH_EXAMPLES=OFF -DWITH_EXTERNAL_BACKEND=ON -DWITH_GCOV=OFF -DWITH_HWMON=OFF -DWITH_IIOD=OFF -DWITH_LIBTINYIIOD=OFF -DWITH_LOCAL_BACKEND=OFF -DWITH_LOCAL_CONFIG=OFF -DWITH_LOCAL_DMABUF_API=OFF -DWITH_LOCAL_MMAP_API=OFF -DWITH_MAN=OFF -DWITH_MODULES=OFF -DWITH_NETWORK_BACKEND=OFF -DWITH_NETWORK_BACKEND_DYNAMIC=OFF -DWITH_SERIAL_BACKEND=OFF -DWITH_SERIAL_BACKEND_DYNAMIC=OFF -DWITH_USB_BACKEND=OFF -DWITH_USB_BACKEND_DYNAMIC=OFF -DWITH_UTILS=OFF -DWITH_XML_BACKEND=OFF -DWITH_ZSTD=OFF
-- cmake version: 3.18.4
-- The C compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test HAS_WPEDANTIC
-- Performing Test HAS_WPEDANTIC - Success
-- Performing Test HAS_WSHADOW
-- Performing Test HAS_WSHADOW - Success
-- Looking for strdup
-- Looking for strdup - found
-- Looking for strndup
-- Looking for strndup - found
-- Looking for strerror_r
-- Looking for strerror_r - found
-- Looking for strtok_r
-- Looking for strtok_r - found
-- Looking for newlocale
-- Looking for newlocale - found
-- Setting -Werror
-- Found Git: /usr/bin/git (found version "2.30.2")
-- Looking for pthread_setname_np
-- Looking for pthread_setname_np - found
-- Features enabled : external
-- Features disabled: threadless compat xml zstd network dns-sd avahi bonjour ipv6 serial local local-dmabuf local-mmap hwmon usb utils examples libtinyiiod iiod modules usb-dynamic network-dynamic serial-dynamic udev-rule doc man man-utils python-bindings c#-bindings c++-bindings
-- LOG_LEVEL set to "Info"
-- Configuring done
-- Generating done
-- Build files have been written to: /home/rgetz/github/libiio/tmp.NwWmZwpzOZ
Scanning dependencies of target iio
[ 17%] Building C object CMakeFiles/iio.dir/buffer.c.o
[ 17%] Building C object CMakeFiles/iio.dir/block.c.o
[ 23%] Building C object CMakeFiles/iio.dir/context.c.o
[ 29%] Building C object CMakeFiles/iio.dir/channel.c.o
[ 17%] Building C object CMakeFiles/iio.dir/attr.c.o
[ 35%] Building C object CMakeFiles/iio.dir/events.c.o
[ 41%] Building C object CMakeFiles/iio.dir/device.c.o
[ 47%] Building C object CMakeFiles/iio.dir/backend.c.o
[ 52%] Building C object CMakeFiles/iio.dir/library.c.o
[ 58%] Building C object CMakeFiles/iio.dir/mask.c.o
[ 64%] Building C object CMakeFiles/iio.dir/scan.c.o
[ 70%] Building C object CMakeFiles/iio.dir/sort.c.o
[ 76%] Building C object CMakeFiles/iio.dir/stream.c.o
[ 82%] Building C object CMakeFiles/iio.dir/task.c.o
[ 88%] Building C object CMakeFiles/iio.dir/utilities.c.o
[ 94%] Building C object CMakeFiles/iio.dir/lock.c.o
[100%] Linking C shared library libiio.so
[100%] Built target iio
Deleted temp working directory /home/rgetz/github/libiio/tmp.NwWmZwpzOZ
-- cmake version: 3.18.4
-- The C compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test HAS_WPEDANTIC
-- Performing Test HAS_WPEDANTIC - Success
-- Performing Test HAS_WSHADOW
-- Performing Test HAS_WSHADOW - Success
-- Looking for strdup
-- Looking for strdup - found
-- Looking for strndup
-- Looking for strndup - found
-- Looking for strerror_r
-- Looking for strerror_r - found
-- Looking for strtok_r
-- Looking for strtok_r - found
-- Looking for newlocale
-- Looking for newlocale - found
-- Found Git: /usr/bin/git (found version "2.30.2")
-- Looking for pthread_setname_np
-- Looking for pthread_setname_np - found
-- The CXX compiler identification is GNU 10.2.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Features enabled : external c++-bindings
-- Features disabled: threadless compat xml zstd network dns-sd avahi bonjour ipv6 serial local local-dmabuf local-mmap hwmon usb utils examples libtinyiiod iiod modules usb-dynamic network-dynamic serial-dynamic udev-rule doc man man-utils python-bindings c#-bindings
-- LOG_LEVEL set to "Info"
-- Configuring done
-- Generating done
-- Build files have been written to: /home/rgetz/github/libiio/tmp.mHISD6UCE4
Scanning dependencies of target iio
[ 21%] Building C object CMakeFiles/iio.dir/context.c.o
[ 26%] Building C object CMakeFiles/iio.dir/block.c.o
[ 26%] Building C object CMakeFiles/iio.dir/buffer.c.o
[ 26%] Building C object CMakeFiles/iio.dir/attr.c.o
[ 21%] Building C object CMakeFiles/iio.dir/backend.c.o
[ 31%] Building C object CMakeFiles/iio.dir/device.c.o
[ 36%] Building C object CMakeFiles/iio.dir/events.c.o
[ 42%] Building C object CMakeFiles/iio.dir/channel.c.o
[ 47%] Building C object CMakeFiles/iio.dir/library.c.o
[ 52%] Building C object CMakeFiles/iio.dir/mask.c.o
[ 57%] Building C object CMakeFiles/iio.dir/scan.c.o
[ 63%] Building C object CMakeFiles/iio.dir/sort.c.o
[ 68%] Building C object CMakeFiles/iio.dir/stream.c.o
[ 73%] Building C object CMakeFiles/iio.dir/task.c.o
[ 78%] Building C object CMakeFiles/iio.dir/utilities.c.o
[ 84%] Building C object CMakeFiles/iio.dir/lock.c.o
[ 89%] Linking C static library libiio.a
[ 89%] Built target iio
Scanning dependencies of target iiopp-enum
[ 94%] Building CXX object bindings/cpp/CMakeFiles/iiopp-enum.dir/examples/iiopp-enum.cpp.o
[100%] Linking CXX executable iiopp-enum
/usr/bin/ld: ../../libiio.a(context.c.o):(.data.rel.ro+0x28): undefined reference to `iio_external_backend'
collect2: error: ld returned 1 exit status
make[2]: *** [bindings/cpp/CMakeFiles/iiopp-enum.dir/build.make:105: bindings/cpp/iiopp-enum] Error 1
make[1]: *** [CMakeFiles/Makefile2:209: bindings/cpp/CMakeFiles/iiopp-enum.dir/all] Error 2
make: *** [Makefile:149: all] Error 2
this is because:
*
* If Libiio was compiled with external backend support (WITH_EXTERNAL_BACKEND),
* applications should implement their own backend and provide the
* iio_external_backend symbol.
*/
extern const struct iio_backend iio_external_backend;
and the cpp bindings dont do that...
I think the same is true when -DWITH_EXAMPLES=ON -DWITH_EXTERNAL_BACKEND=ON is set (more build errors).
libiio declares the variable iio_external_backend but intentionally does not define it, leaving the responsibility for the client code to provide its definition.
So far, I thought of the following:
Option A: Modify the mechanism that allows clients to define their own external backend. However, I haven't yet found a cross-platform solution that works well for both static and dynamic builds of libiio.
Option B: Accept that not all combinations of available CMake options can produce a valid build. Some options inherently conflict with others—for example, enabling NO_THREADS conflicts with all current backends (local, serial, USB, network), requiring them to be disabled. Similarly, enabling WITH_EXTERNAL_BACKEND would necessitate disabling examples, utils, C++ bindings, and iiod, as all these generate libiio-based executables, which require a definition for iio_external_backend at link time. However, this approach prevents building examples and utilities that would use a custom external backend.
Option C: Introduce a new CMake variable that allows users to specify a .h/.c file where iio_external_backend is defined. The repository could even provide a default definition for cases where we simply want to ensure the build process completes successfully.
why not just define a default weak symbol (that is nop/null), that can be overridden externally?
-Robin
I defined it as weak symbol and created a small test app that overrides it. Within the test app I can see the overridden version of the symbol but the libiio library uses the symbol within and that version is the nop/null version. As a conclusion this approach will fix the build error but will also produce a libiio binary that contains the wrong version of iio_external_backend. This is how I defined the weak symbol:
diff --git a/backend.c b/backend.c
index 835a8065..896ca525 100644
--- a/backend.c
+++ b/backend.c
@@ -12,6 +12,15 @@
#include <string.h>
+__attribute__((weak))
+const struct iio_backend iio_external_backend = {
+ .api_version = 0,
+ .name = "default-weak-backend",
+ .uri_prefix = NULL,
+ .ops = NULL,
+ .default_timeout_ms = 0,
+};
+
unsigned int iio_get_builtin_backends_count(void)
{
unsigned int i, count = 0;
And this is the test app:
#include <iio/iio.h>
#include <iio/iio-backend.h>
#include <stdio.h>
#include <string.h>
// Override the weak symbol
const struct iio_backend iio_external_backend = {
.name = "custom-override-backend"
};
int main(void) {
printf("Backend name: %s\n", iio_external_backend.name);
int nb_b = iio_get_builtin_backends_count();
printf("builtin backends count: %d\n", nb_b);
int i;
for (i = 0; i < nb_b; i++) {
printf("backend with index: %d has name: %s\n", i, iio_get_builtin_backend(i));
}
return 0;
}
which prints out:
Backend name: custom-override-backend
builtin backends count: 1
backend with index: 0 has name: default-weak-backend
And the problem is that it should have printed "custom-override-backend" instead of "default-weak-backend".
Next, I tried to use a function to return the symbol which in theory it should work. But because libiio adds the symbol like this:
https://github.com/analogdevicesinc/libiio/blob/53c316018462535a562164fed4b4035f9c9c8a61/context.c#L402
I get the following error:
error: initializer element is not constant
Because the function approach doesn't provide a constant expression an alternative would be somehow initialize iio_backends[] at runtime.
Are all these potential changes really justified, considering the end goal?
You just need to poke a little more using readelf or nm to see where the problem is. Taking your patches from the above:
If you look at
readelf -Ws ./CMakeFiles/iio.dir/backend.c.o | grep iio_external_backend
23: 0000000000000000 40 OBJECT WEAK HIDDEN 6 iio_external_backend
you can see the object is weak in the local compilation object (which is good) - but when it's linked:
readelf -Ws ./libiio.so | grep iio_external_backend
229: 0000000000026480 40 OBJECT LOCAL DEFAULT 23 iio_external_backend
it got promoted to a strong symbol This is bad. :(
This is because we use (in the CMakeLists.txt):
if (CMAKE_COMPILER_IS_GNUCC)
206 if (NOT WIN32)
207 target_compile_options(iio PUBLIC -fvisibility=hidden)
208 endif()
the visibility=hidden affects weak symbols, (when compiled with -fvisibility-hidden, the symbols no longer are weak) this can be fixed by using this:
extern const struct iio_backend __attribute__((weak, visibility("default"))) iio_external_backend;
this should give you"
libiio.so
271: 0000000000026480 40 OBJECT WEAK DEFAULT 23 iio_external_backend
366: 0000000000026480 40 OBJECT WEAK DEFAULT 23 iio_external_backend
and make things work for you. I got:
Backend name: custom-override-backend
builtin backends count: 5
backend with index: 0 has name: local
backend with index: 1 has name: ip
backend with index: 2 has name: usb
backend with index: 3 has name: xml
backend with index: 4 has name: custom-override-backend
The trick will be making to work across all the compilers that are supported. (limiting external back-ends to certain compilers might be a good idea anyway).
-Robin
PS - I did run into a problem when compiling inside libiio, when all iio_external_backend functions are defined as weak. Since we also have SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-Bsymbolic") set - which causes the library to be searched first - to handle the compatibility layer... If you are going to test things within the libiio build environment - as a different utility or something - you need to comment out that line in the cmake (or fix it so it only effects the library, not the utils, or examples - which might be a nice way to document/test this).
Based on my tests, I am not getting this:
backend with index: 4 has name: custom-override-backend
but
backend with index: 4 has name: default-weak-backend
This happens when -Wl,-Bsymbolic flag is applied. Commenting it out solves the issue.
I am seeing this both within the libiio build environment and in external client code that links against libiio.
It seems that -Wl,-Bsymbolic instructs the linker to bind all internal references to symbols inside the shared object. Because of this, the weak implementation will be always used.
The LIBIIO_COMPAT option requires -Wl,-Bsymbolic to be enabled. This means LIBIIO_COMPAT and WITH_EXTERNAL_BACKEND can't be enabled at the same time.
One idea that comes to drop the mind is to drop the weak symbol mechanism add a setter in the public API that allows users to provide a reference to a custom backend. If this setter isn't called, libiio would simply hold a null reference. This approach would:
- Eliminate the need for the WITH_EXTERNAL_BACKEND option
- Ensure compatibility with compilers that don't support weak symbols (such as MSVC)
- Require client code to call the setter early, before making most libiio calls
This happens when -Wl,-Bsymbolic flag is applied. Commenting it out solves the issue.
yes - in my previous tests, I just commented it out.
I think there is a way to do this with weak symbols ; using --dynamic-list which can override (is my understanding) -Bsymbolic - but it will take some more testing. I can have a look tomorrow I think.
One idea that comes to drop the mind is to drop the weak symbol mechanism add a setter in the public API that allows users to provide a reference to a custom backend. If this setter isn't called, libiio would simply hold a null reference. This approach would:
- Require client code to call the setter early, before making most libiio calls
but this would only be in the case that someone had an external back end - correct? in most cases it would be set to null, and a nop if not set.
I don't know how @pcercuei envisioned people to use this? We didn't talk about it before he left.
I don't know how @pcercuei envisioned people to use this? We didn't talk about it before he left.
Pretty much how Dan said, it's supposed to be used in the case where the client application provides the backend. Typically on microcontrollers where the backend is in a separate library.
I don't see any use case where you'd want to build an application with libiio built with WITH_EXTERNAL_BACKEND=ON and not providing the external backend. In that case providing a weak default symbol is asking for trouble, as you could forget to link the backend or name it incorrectly and it would still compile successfully. Also, remember that weak symbols is very much a GCC/Clang thing, and it wouldn't work on anything else (including Windows DLLs).
The problem is - as it exists - when you turn external backend on - we can't build with examples since they don't provide a backend...
We can force those off - but then no distribution can ever use it - so I'm not sure I see the value.
- Robin
The reason it was introduced was to support Libiio + external backend + thread-less IIOD as a replacement for tinyiiod. IMHO it shouldn't be enabled by default for any distribution - there's just no use outside the microcontroller space.
Then I'm even more confused.
If external backends are meant to replace tinyiiod why was it added in the same change set? https://github.com/analogdevicesinc/libiio/pull/1128
or you mean libtinyiiod - the separate repo?
If external backends are meant to be dependent on the internal tinyiiod (for non-Linux environments like MCU) , and hidden otherwise - then we should describe it like that in the build readme, and enforce that in the cmake and start deprecating the libtinyiiod separate repo.
Thanks -Robin
Then I'm even more confused.
If external backends are meant to replace tinyiiod why was it added in the same change set? #1128
or you mean libtinyiiod - the separate repo?
I mean the separate repo, yes.
Paul - thanks for the clarification. Things are slowly coming together.
So to address the original described issue. We should:
- disallow external backends unless tinyiiod is enabled (cmake change)
- when building with the internal tinyiiod (for MCUs) we should turn off examples, and utils (which are all Linux based). In this case - that would mean adding another Cmake flag for the C++ examples. (In case you wanted to do C++ on your MCU based on FreeRTOS or Zephyr).
- add some separate CI/CD to build test in the MCU configurations
- split POSIX (Linux/BSD/Windows) configurations and MCU configurations out in the build readme.