cereal
cereal copied to clipboard
USCiLab
Add MessagePack archiver
A MessagePack (http://msgpack.org/) archiver would be great. This would have the advantage of being as cross-platform as the json archiver, however with much smaller memory requirements...
Any archives included within cereal need to be header only including all dependencies so that we can bundle all of them.
We won't implement support for MessagePack ourselves but would accept a well written archive into cereal. Here are a few C++ implementaions for anyone interested: https://github.com/msgpack/msgpack-c and https://github.com/Lichtso/netLink.
Sounds good. Yes, I know the two implementations. However, they are both not header only. I see two options:
- Just add the archiver header to cereal and require the user to have message pack already installed
- also add one of the two message pack libraries to cereal, and make them header only by converting all functions to inline functions.
What do you think?
Option 1 is not a valid option for inclusion in cereal - you are free to develop and release this standalone, but anything included in cereal itself must be entirely self contained.
Option 2 is feasible depending upon the license of the libraries and how large they are - ideally we want anything related to an archive to be as small as possible. A message pack archiver would likely be folded into some kind of a module for cereal, see #123, unless the implementation was concise.
Comment: This would be option 2. The library is (in the newest version on github) header only and not that big. Also the license is Apache LicenseVersion 2.0, which should be fine I guess?
I discussed this with @randvoorhies and we will very likely end up merging the code but it will become a cereal module, which is something we're going to roll out for 1.2. We'll be making repositories for the various modules at https://github.com/cerealcpp while keeping the main cereal repository in the same place it is now.
In regards to your code, the biggest impact this will have is that we are going to restructure unit tests a little - right now the unit tests are monolithic source files - we'll split up the actual test into a header file (e.g. the test_array() function) which can be included and instantiated as necessary (e.g. BOOST_TEST_CASE( xxx )). This way you don't need to modify the tests that ship with cereal and your module will ship with its own test suite. Or potentially we could have some kind of a system where the build system would know about your custom archive and add it to the tests.
We'll have to figure out the best way to market the modules, I'm guessing for now we'll hand curate them and list them on the website.
Sounds good. Yes, I know the two implementations. However, they are both not header only.
@rggjan, msgpack-c is indeed header-only if you use the C++ API
@ecorm, yes I know, that's great! At the time of writing this was not the case yet, but msgpack-c added header only support to the newest version.
The pull request is at #138, we haven't made any progress in merging it yet.
Having a cross language binary format (any of them) is very desirable for me. I'm going to try to work from the pull request fork until it's merged.
How does this perform, memory wise, compared to the included Binary and Portable Binary archives? Does MessagePack output smaller archives?
How about https://github.com/mikeloomisgg/cppack? It is a single header so much simpler than msgpack-c. The interface is also quite similar to cereal so maybe it would be easy to do. One disadvantage is that the last commit was from 2019.
@stephanlachnit When it works, it works, so that may be a good option for some people. I've been using Rust with serde, lately, and I've translated most of the C and C++ that I modify frequently to Rust, so it worked out well. The backends for serde give a variety of formats, and it doesn't require any more than an annotation at the top of a struct to make it work.