Standardize ABI & SONAME

[paravoid]

Motivation

In order for one to be able to embed WasmEdge in their application and expect the app to not break as newer WasmEdge versions are installed, it would be useful for WasmEdge to standardize on an ABI, and add a SONAME version to freeze that ABI.

containers/crun@33f900cc3feec6e9348fff16204e11795d28e12f is an example of an (API & ABI) breakage that as I understand it occured with 0.10.0, that would have likely resulted in crun not working after a WasmEdge upgrade. (Note: I am not affiliated with the crun project or author of that commit, so this is my own interpretation). Having a way to have e.g. both "libwasmedge_c.so.0" and "libwasmedge_c.so.1" present on the system would had allowed for both to be present, and a smooth upgrade path to be provided.

This would be also useful for distributions, where SONAME Is typically embedded into the package name to allow co-installability.

Details

Adding a SONAME version suffix is trivial with CMake, e.g.

diff --git a/lib/api/CMakeLists.txt b/lib/api/CMakeLists.txt
index 459ca274..830e8319 100644
--- a/lib/api/CMakeLists.txt
+++ b/lib/api/CMakeLists.txt
@@ -84,6 +84,8 @@ if(WASMEDGE_BUILD_SHARED_LIB)
 
   set_target_properties(wasmedge_c_shared PROPERTIES
     PUBLIC_HEADER "${WASMEDGE_CAPI_HEADERS}"
+    VERSION ${CMAKE_PROJECT_VERSION}
+    SOVERSION ${CMAKE_PROJECT_VERSION_MAJOR}
     MSVC_RUNTIME_LIBRARY "MultiThreadedDLL"
   )

This assumes CMAKE_PROJECT_VERSION exists, which was my main motivation for the -flawed as of this moment- #1649. This also assumes that the SOVERSION would be the major version ("0" right now), and hence wouldn't have prevented the 0.9.0 -> 0.10.0 breakage.

The real change is a policy one: at which point is the ABI considered "stable", versioned, and by which version. The CMake machinery is easy to adjust otherwise.

Appendix

While considering that, it may be a good time to consider renaming libwasmedge_c.so to just libwasmedge.so (and thus with the change above, also libwasmedge.so.0 etc.). The "_c" portion is a bit odd and superfluous, as C calling conventions (as opposed to e.g. C++ mangled) are usually the standard ones in all OSes.

[hydai]

Hi @paravoid Thanks for raising this. This is a very helpful idea.

We found that we have to make other breaking changes in the WasmEdge C API because there are some important features including extending the return error code section for the host functions and plugins.

And we can also expect there may be more breaking changes in the future, especially before the 1.0 releases. After considering the possible cases, we will have libwasmedge.so.0.0 in the next release 0.11.0, and also remove the _c postfix.

To prevent the SOVERSION 0 cannot present the breaking changes between different 0.x.y. We decide to provide another version for the C ABI starting from 0.0. Once there is any breaking change, then we will have SOVERSION 1.0 to indicate that. I believe this can cover all the situations.

I am working on this. Please feel free to give me any additional recommendations or concerns.

[paravoid]

Thank you for considering this, appreciated!

I think that all makes sense. Bumping the SONAME on breaking changes would address the use cases I'm talking about before.

I'm only a little confused by seeing the SONAME being "libwasmedge.so.0.0". Typically you would see a singular number in the SONAME (e.g. "libwasmedge.so.0"), and the filename being the full version string (libwasmedge.so.0.11.0) with the SONAME being a symlink to it (libwasmedge.so.0 -> libwasmedge.so.0.11.0).

Finally, over at #1678 I proposed a version script. Another option you could explore here for ABI changes is symbol versioning, where you can evolve the ABI without breaking compatibility (in short: the same symbol can have a different prototype depending on the version). Of course this shifts some maintentance overhead to you from your downstream users, so it may or may not be appropriate depending on how volatile the ABI is at the moment.

Hope this all helps!

[hydai]

Ah, it must be a typo.

We will have the following structure:

libwasmedge.so -> libwasmedge.so.0
libwasmedge.so.0 -> libwasmedge.so.0.0.0
libwasmedge.so.0.0.0

The SONAME will be libwasmedge.so.0. The SOVERSION will be libwasmedge.so.0.0.0.

PR: #1783

[hydai]

Checklist:

• [ ] Update the WasmEdge handler in crun after the 0.11.0 release.
• [x] Update the WasmEdge-go
• [x] Update the WasmEdge-sys, maybe WasmEdge-sdk are also needed to update.
• [x] Notify the maintainer of the installer to make sure there will be fine if the so name changed.

[hydai]

Hi @SAtacker In the release of 0.11.0, we will make a breaking change for the C API library which is mentioned in this issue. The libwasmedge_c.so is now called libwasmedge.so and it will be a symlink to libwasmedge.so.0 which will be handled by the CMake, so you don't need to do symlink in the installer. The libwasmedge.so.0 will be a symlink to libwasmedge.so.0.0.0, which will also be handled by the CMake. However, I am not sure if the current install can handle this or not. Could you please check it?

Since the PR is still in the review process, you can use an existed pre-built tarball, do the rename and symlink, and tar them again to compose a new tarball for testing. Thanks.

[paravoid]

Ah, it must be a typo.

We will have the following structure:

libwasmedge.so -> libwasmedge.so.0
libwasmedge.so.0 -> libwasmedge.so.0.0.0
libwasmedge.so.0.0.0

The SONAME will be libwasmedge.so.0. The SOVERSION will be libwasmedge.so.0.0.0.

PR: #1783

Ah, perfect. That makes sense! Thank you!

Any thoughts on the second part of my message above? Sorry if I'm being insistent -- #1678 remains in a closed status (and I can't reopen) despite the leaked symbols, so trying to make sure we don't drop the ball here as progress is being made on ABI standardization! Thanks again!

[hydai]

Ah yes. However, we still need those symbols to make the plugin system work. So we would like to keep them currently until we figure out a new way to handle the plugin system. A new alternative plugin system is on the way. If the development is done and we decide to deprecate the current plugin design, then this will be fixed at that time.

[SAtacker]

Hi @SAtacker In the release of 0.11.0, we will make a breaking change for the C API library which is mentioned in this issue. The libwasmedge_c.so is now called libwasmedge.so and it will be a symlink to libwasmedge.so.0 which will be handled by the CMake, so you don't need to do symlink in the installer. The libwasmedge.so.0 will be a symlink to libwasmedge.so.0.0.0, which will also be handled by the CMake. However, I am not sure if the current install can handle this or not. Could you please check it?

Since the PR is still in the review process, you can use an existed pre-built tarball, do the rename and symlink, and tar them again to compose a new tarball for testing. Thanks.

Alright, will let you know in a few minutes, thanks for notifying me.

[hydai]

Alright, will let you know in a few minutes, thanks for notifying me.

FYR, I use the cp -rp to make sure the soft link is fully copied by the cp command. Hope this may help.

[SAtacker]

Alright, will let you know in a few minutes, thanks for notifying me.

FYR, I use the cp -rp to make sure the soft link is fully copied by the cp command. Hope this may help.

I am using the new installer, almost ready. Should be alright in a few minutes.

[hydai]

Hi @paravoid Thanks for your assistance. The 0.11.0 is out, and the SONAME is shipped. To track the non-WasmEdge symbol exported issue, I reopen that one, so we can have the right place to discuss the further details.

[hydai]

Since this feature is shipped in 0.11.0, I am closing this. Thanks!