cereal
cereal copied to clipboard
detail namespace collision with nlohmann::json?
I'm seeing a very strange bug when using your library in conjunction with the newest version of https://github.com/nlohmann/json.
Reproduce with the following:
#include <json.hpp>
#include <cereal/types/polymorphic.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/access.hpp> //So we can make serialize private so developers aren't tempted to call it.
#include <cereal/types/string.hpp> //This is needed to serialize std::string. There are similar ones for the other std containers
#include <cereal/types/vector.hpp> //This is needed to serialize std::vector. There are similar ones for the other std containers
using namespace std;
//Pure virtual base class
class Serializable
{
public:
virtual int getType() = 0;
protected:
template<class Archive> void serialize(Archive & ar);
};
class Bug: public Serializable
{
public:
std::string text;
int getType() {return 1;};
void load(const nlohmann::json & s) {};
private:
friend class cereal::access;
template <class Archive> void serialize(Archive &ar) {ar(text);};
};
// Register Bug
CEREAL_REGISTER_TYPE(Bug);
CEREAL_REGISTER_POLYMORPHIC_RELATION(Serializable, Bug);
int main()
{
auto obj = make_shared<Bug>();
std::ostringstream os;
{
cereal::BinaryOutputArchive oarchive(os);
oarchive(dynamic_pointer_cast<Serializable>(obj));
}
shared_ptr<Serializable> obj2;
std::istringstream is(os.str());
{
cereal::BinaryInputArchive iarchive(is);
iarchive(obj2);
}
auto m = dynamic_pointer_cast<Bug>(obj2);
return 0;
}
This code, when compiled with the latest nlohmann::json
throws the following:
Trying to load an unregistered polymorphic type (Bug).
Make sure your type is registered with CEREAL_REGISTER_TYPE and that the archive you are using was included (and registered with CEREAL_REGISTER_ARCHIVE) prior to calling CEREAL_REGISTER_TYPE.
If your type is already registered and you still see this error, you may need to use CEREAL_REGISTER_DYNAMIC_INIT.
Curiously, though, it only happens when the name of the function in the Bug
class is load
, which is why I'm opening an issue with your library (it sseems to have special significance with your library).
Again, curiously, compiling against a 2.X version of nlohmann::json
does not product this exception.
I noticed one change from nlohmann::json
2.X->3.X is the inclusion of the nlohmann::detail
namespace. Could this be somehow conflicting with cereal::detail
? I will also open an issue against nlohmann::json
.
Still trying to figure out root cause
nlohmann::json issue: https://github.com/nlohmann/json/issues/1082
So this only triggers at runtime on that polymorphic check? Do you have any issues with non-polymoprhic code when using both libraries?
It looks like I also have a problem with non-polymorphic code:
#include <json.hpp>
// #include <cereal/types/polymorphic.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/access.hpp> //So we can make serialize private so developers aren't tempted to call it.
#include <cereal/types/string.hpp> //This is needed to serialize std::string. There are similar ones for the other std containers
#include <cereal/types/vector.hpp> //This is needed to serialize std::vector. There are similar ones for the other std containers
using namespace std;
class Bug
{
public:
std::string text;
void load(const nlohmann::json & s) {};
private:
friend class cereal::access;
template <class Archive> void serialize(Archive &ar) {ar(text);};
};
int main()
{
Bug obj;
std::ostringstream os;
{
cereal::BinaryOutputArchive oarchive(os);
oarchive(obj);
}
Bug obj2;
std::istringstream is(os.str());
{
cereal::BinaryInputArchive iarchive(is);
iarchive(obj2);
}
return 0;
}
Has the following static assertion:
static assertion failed: cereal found more than one compatible input serialization function for the provided type and archive combination.
Types must either have a serialize function, load/save pair, or load_minimal/save_minimal pair (you may not mix these).
Use specialization (see access.hpp) if you need to disambiguate between serialize vs load/save functions.
Note that serialization functions can be inherited which may lead to the aforementioned ambiguities.
In addition, you may not mix versioned with non-versioned serialization functions.
However, if I either:
- Change the name from
Bug::load
to something else likeBug::loadJson
- Roll back
nlohmann::json
version
It magically works
I've also confirmed that the break occurs when moving from nlohmann::json
version 2.0.10
-> 2.1.0
. Version 2.1.0
introduces the nlohmann::detail
namespace.
Update: I've confirmed it is definitely not a namespace collision. I renamed detail to an arbitrary name and I still see the same problem. Instead, it looks like somehow Cereal thinks the nlohmann::json type is a cereal::Archive such that when passed into a function named load it thinks I am registering a new load/save function. Unfortunately both libraries are extremely template heavy so it's hard to understand what's going on
Has a fix been found this problem? I'm seeing the same in my code (with the difference being that I am writing load and store functions to serialize a nlohmann::json
into an archive). Cereal finds multiple serialization functions. I cannot find a load, store, or serialize function in the nlohmann-json code that could conflict, though...