Feature Request: Support creating a shared library

[Radagan]

I'm using this header only library under Linux running on arm64 and the memory use greatly increases compile time on the board. While it's possible to cross-compile, just having a library to link to would solve the problem.

Has this been considered?

[jonesmz]

You're talking about having a library that you can download from here, on github?

I don't believe that there's really any plans to do this, since the number of possible platforms that would need to be compiled for by this project could get quite large, especially considering the various libc implementations that the different embedded platforms target, not to mention compatibility with different boost library versions, shared linking versus static linking, and so on.

Ultimately it gets nuts really fast.

What could be done is set up a CMake target that creates the shared library for you on your system, and then you can link against that.

A fairly straight forward way to do this is to define some header file

#include <your list of mqtt_cpp headers here>
extern template *the various types you want compiled into your shared library here*

and then have a cpp file that explicitly instantiates the types that you defined as an extern template.

Then include your new header file so that the compiler knows you want to use your definition of the template types from the shared library, instead of from your current translation unit.

I'm sure that if you wanted to create a pull request that adds this as an optional (e.g., only happens if cmake is given option to turn it on) feature, that it would be something @redboltz is interested in.

[Radagan]

What you describe is exactly what I'm looking for. I have no interest in downloadable libraries, just to build them from the mqtt_cpp source, and then link to them in the usual way.

Unfortunately, I don't quite follow your advice on how to do it. I'm just learning CMake. Can you give me a few more breadcrumbs? I have a strong interest in this project and would like to use it in more memory constrained environments like Raspberry Pi. Not embedded environments, but Linux based single board computers. They are fast enough to simply use, but typically have about 1G of main memory, so compiling with this header library hits the swap file immediately.

I don't mind helping, but will need a bit more guidance to get started. I work in commercial building automation, and having access to an MQTT library committed modern C++ and built on boost was a fantastic find!

[jonesmz]

I'm using this project on a Linux board that has under 1GB of ram, and runs OpenWRT.

But I'm using a cross compiler environment to build for it, so my constraints are slightly different.

How familiar are you with C++? Do you know how template instantiation works, and how the linker handles things when combining multiple different .o files?

[Radagan]

I started off with C++ many years ago, but have only recently come back to it. I've been writing cloud and mobile apps since around 2008, so mainly Java and Objective-C (iOS), and recently Dart/Flutter. I somehow missed the incredible additions to C++ that happened in 2011 until only recently. So when an opportunity to work again in C++ came up, I jumped on it. I'm loving it, but you are correct in your thoughts that I have significant gaps in my knowledge--I'm working on them though.

I understand templates in a generic programming sense, but not how they would relate to how a linker would handle the object code the compiler would generate. I'm currently using clang++ if that's important.

As an aside, do you know a good primer on how to setup a cross compiler? I've found a few "magic" how-tos, but nothing that has details. I know clang has built in capabilities, but setting it all up in cmake is complex--that is if you don't already know how. I'm currently targeting arm64 (Ubuntu 19.04 on Raspberry Pi 3Bs).

[jonesmz]

Actually, I don't really know anything about how to set up a cross compiler.

I'm using the OpenWRT build root, which has all the "magic" built in already, so I didn't have to do any configuration (lucky me!)

My advice in terms of setting up CMake to do things in more "chunks" (Saving on the ram needed for each part of the compile process) is that you can use "extern templates", which was a language level feature that was added in C++11.

Imagine you have a C++ template (function, class, whatever) defined in some header file X.h

Normally, any place where you use the template you defined in X.h will cause the compiler to instantiate the template with the various template parameters you provide.

With extern templates, you would define your template type like normal, and then immediately after, declare the template type as "extern".

E.g.

template<typename T>
class Foo { };

extern template class Foo<int>;

Then, in some C++ file (e.g. X.cpp), you must explicitly instantiate the type that you declared extern.

#include <X.h> // for the declaration of Foo<T>
template class Foo<int> // Explicitly force instantiation, ignoring the `extern` keyword from X.h

Note also that you must do this same dance for each specific version of the template you want instantiated. If you want Foo and also Foo you must do both separately, which really isn't a big deal since it's only about 2 lines of code for each additional extern template variation you want.

What this basically boils down to is:

I, the programmer, do solemnly swear, that this template type will exist at link time, and therefore you, the compiler, shall not instantiate this template type in this translation unit, unless I explicitly tell you to.

Which, well, prevents the compiler from instantiating the template type, even though you're using it, and therefore, the compiler does less work, and thusly uses less ram.

If you don't have a cpp file (which gets compiled into a .o file, of course, and then linked like normal) that explicitly instantiates the template type, then you've lied to your compiler and your program won't link (undefined symbols referenced by blahblahblah...)

Do note: The majority of stack overflow posts about this subject explicitly show the extern template keywords being used in the cpp file that uses your .h file, meaning that you would need to opt in for every single cpp file you have. Which is stupid, in my opinion. Either every instance of the type should be extern (save the explicit instantiation you provide) or none of them. Mixing and matching just results in the compiler doing a lot of work that the linker will throw away.

So you can take advantage of this behavior to create an optional CMake feature for this project, where when the feature in question is enabled, extern template will be used for the various template types defined by mqtt_cpp (or at least the big ones), and a new shared library (Maybe a static library would be a better call?) will be created by CMake that's made up of one or more cpp files that explicitly instantiate the template types that were declared as extern templates.

I can provide some minor CMake examples on how to create a new library target, but my CMake foo is not very strong either, so it might be a case of the blind leading the blind.

[Radagan]

That is an amazing feature of C++11! Thank you very much for taking the time to explain it. I am studying the latest Stroustrup, but I might never have caught on to that feature of templates.

Okay, I must try this... I'll take whatever CMake foo your willing to suggest and take it from there. I have two CMake books on my "read now" list as well. I'm really impressed with CMake, but I'm a neophyte.

[jonesmz]

Well, here's a trivial example that links against the mqtt_cpp "library". You can use that to get started.

Other libraries can link against this one by doing:

TARGET_LINK_LIBRARIES(myproject_myprogram myproject::mqtt)

PROJECT(myproject_mqtt)

MESSAGE(STATUS "Configuring ${PROJECT_NAME}")

FILE(GLOB SRC "*.cpp")

ADD_LIBRARY(myproject_mqtt ${SRC})
ADD_LIBRARY(myproject::mqtt ALIAS myproject_mqtt)

TARGET_INCLUDE_DIRECTORIES(myproject_mqtt PUBLIC /path/to/my/includes/)

TARGET_LINK_LIBRARIES(myproject_mqtt myproject::core myproject::log mqtt_cpp_iface Boost::system OpenSSL::SSL nlohmann_json::nlohmann_json )

IF(SOME CONDITION)
    ADD_SUBDIRECTORY(test EXCLUDE_FROM_ALL)
ENDIF()

[Radagan]

Thank you! This helps.

[Radagan]

This article is interesting and describes exactly what we're talking about: a library that offers static, dynamic, and header only options.

https://steveire.wordpress.com/2016/08/09/opt-in-header-only-libraries-with-cmake/

To use mqtt_cpp, I think I'm going to have to try something like this. I tried building with an async client and the build ran out of memory even with a swap file. It's that or cross compile I guess. I really don't want to have to do that though.

[jonesmz]

@redboltz Note also that improvements to this situation may also make the project's CI run faster.

I notice that a full run of Travis CI currently takes > 30 minutes, because a lot of the same code is being compiled over and over again.

Setting up precompiled headers (even if just for continuous integration), and having optional shared libraries would speed Travis CI and AzurePipelines up a lot.

[redboltz]

I used to try cotire https://github.com/sakra/cotire and ccache. However no significant speed up is observed at that time. Basically I don't want to change source code for this kinds of feature, but updating build tree is no problem. Maybe when I tried that, I might be missing something. I'm not sure how PCH like mechanism work well template base library especially function template infer based one. So I choose every time compile. At least this approach introduce no troubles except compile speed.

If someone try this again and send PR, it is welcome.

[Radagan]

I am still very interested in working on this features as my compile times are almost prohibitive on Raspberry Pi 3B+. I've been quiet lately because I'm moving from Australia to the U.S., but I'm willing to chip in on this in a month or so when the move is over.

[redboltz]

@Radagan , your contribution is always very welcome :)

BTW, you can cross-compile your application that includes header-only mqtt_cpp. Here is example of RaspberryPI cross compiler building files:

https://github.com/kikairoya/raspi-tools

It works on Arch Linux.

[redboltz]

I just googled docker environment and found https://github.com/sdt/docker-raspberry-pi-cross-compiler.

[jonesmz]

@Radagan

I rewrote a lot of the cmake configuration recently.

Did that make the problem any better for you?

[Radagan]

@jonesmz

Thank you. I've been relocating from Australia to the U.S. the past couple of months and haven't been able to work on this project again yet. But I will try a compile very soon. My office equipment made it in yesterday and I'm sorting it all out. Will definitely let you know!

[jonesmz]

@Radagan Welcome to the U.S.

Have you settled in well?

Do you think you could revisit this issue at some time?

[Radagan]

@jonesmz Thank you! It's good to be home, but still in a bit of transition. Moving from tropical Queensland to Georgia in the Winter after seven years is a big change.

Other work has kept me off this project thus far, but I am committed to working with this library. I need a clean, modern C++ MQTT library. I'm willing to help out here. Give me a couple of weeks as I'm up to my ears in a Flutter app at the moment, then I can give some feedback.

I probably should just setup a cross-compile tool chain, but I'll definitely dust off and try rebuilding my current RPi based solution first.