[WIP]revive the export_import example

[PengZheng]

I'm studying the OSGi lifecycle layer using this example. If there is any refactoring work needed for it, please let me know.

[pnoltes]

Hi PengZheng,

Nice to see you working on the export / import example. The whole import/export library functionality for Celix has been dormant for quite some time.

The idea was of course that this should mimic the Export-Package / Import-Package of OSGi Module Layer. Only in C there are no runtime packages, so the most logical thing to mimic the java packages was shared libraries.

So bundles (zip files) with shared libraries (so/dylib files) and using dynamic library loading (dlopen) should make this possible?. Well... the details are important. For example. Say you have a library foo and you have two bundles (A and B) exporting a version of this library (1.8.0 and 2.0.0). You also have two bundles (C and D) importing these libraries. C wants version [1,2) (so version 1 and up, but not 2 or higher than 2) and D wants version [2,3). So

Bundle A  -- Export --> libfoo.so 1.8.0
Bundle B  -- Export --> libfoo.so 2.0.0
Bundle C <- Import --- libfoo.so [1,2)
Bundle D <- Import --- libfoo.so [2,3)

For this to really work we need to arrange quite some things. And to keep it simple I just focusing on Linux.

SONAME

The SONAME (field of data in a shared object file) for libfoo.so 1.0.0 and libfoo.so 2.0.0 need to be different. Even when using dlopen with RTLD_LOCAL, dlopen will just reuse symbols already loaded if you load an already loaded library with the same SONAME.
This can be done by ensuring user will compile a shared library with a good unique (for every version) SONAME or by runtime changing the SONAME in a (not yet loaded) library file. IMO the first one makes using Celix too complex and the second one is just too much hacking for my taste.

So after loading, something like

Bundle A  -- Export --> libfoo.so 1.0.0 [SONAME=libfoo.so.1.8.0]
Bundle B  -- Export --> libfoo.so 2.0.0 [SONAME=libfoo.so.2.0.0]

NEEDED

A bundle importing a library make this known by the NEEDED field (again a field of data in a shared object file). This fields needs to be updated to the actual library imported. So a NEEDED with value libfoo.so will need to transferred to a library reference with an actual version.

In this case this cannot be done during compilation, because the whole point in that you runtime import a library which matches your Import statement. So this must be done by updating the NEEDED just before loading the bundle library. And again IMO this is too complex and too much hacking.

So after loading, something like

Bundle C <- Import --- libfoo.so [1,2)  [NEEDED=libfoo.so.1.8.0]
Bundle D <- Import --- libfoo.so [2,3) [NEEDED=libfoo.so.2.0.0]

dlmopen to the rescue

A few years ago (I think 3) dlmopen was added to GNU libc (linux!). If I read the documentation correctly this should make it possible to create a load library namespace and load libraries only into that namespace. Theoretically this should make it possible to create a library namespace for every bundle and then "just" load libraries into the namespace and make exporting/importing possible that way. I did some experiments with this, but at that time dlmopen did not seem stable.

Also dlmopen is only available for linux.

Is it worth it?

All that being said. I am not sure if it is worth it. In my "day job" we do not use import/export of library.

We do use LOCAL loading of private bundle libraries to be able to handle multiple versions of libraries. But even for that use case, I am not sure if dynamic loading of libraries is worth it. The reasons is that dynamic loading of libaries (in this case using bundles) make debugging more diffcult. Especially when a) using different version of libraries with the same symbols and b) post mortem analysing using a core dump. In the latter you need to somehow load the bundle libraries again to get correct stacktraces.

Al this in combination that we are moving to smaller executables running as docker/OCI containers I think the whole module layer of OSGi is not worth it. But that just my opinion. And note that Celix is 'marketed' as a OSGI implementation in C and the module layer (including import/export) is part of the OSGi spec.

Refactoring export/import and wiring

Celix also has some implementation for the OSGi Capability Requirement Model. This part of the Celix has been very dormant for the last years and something I am not really familiar with. The whole Capability Requirement Model is tightly coupled with export/import support. So if this is touched it would be good to revisit the wiring and resolving code of Celix (at least ensure that the types get a celix_ prefix).
See https://github.com/apache/celix/blob/master/libs/framework/include/wire.h https://github.com/apache/celix/blob/master/libs/framework/include/requirement.h https://github.com/apache/celix/blob/master/libs/framework/include/capability.h

Sorry for this dump of information. But I think it is smart to have a good discussion about this, before starting to code.