cereal icon indicating copy to clipboard operation
cereal copied to clipboard

detail namespace collision with nlohmann::json?

Open RPGillespie6 opened this issue 6 years ago • 5 comments

I'm seeing a very strange bug when using your library in conjunction with the newest version of https://github.com/nlohmann/json.

Reproduce with the following:

#include <json.hpp>

#include <cereal/types/polymorphic.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/access.hpp>       //So we can make serialize private so developers aren't tempted to call it.
#include <cereal/types/string.hpp> //This is needed to serialize std::string. There are similar ones for the other std containers
#include <cereal/types/vector.hpp> //This is needed to serialize std::vector. There are similar ones for the other std containers

using namespace std;

//Pure virtual base class
class Serializable
{
    public:
        virtual int getType() = 0;

    protected:
        template<class Archive> void serialize(Archive & ar);
};

class Bug: public Serializable
{
    public:
        std::string text;
        int getType() {return 1;};
        void load(const nlohmann::json & s) {};

    private:
        friend class cereal::access;
        template <class Archive> void serialize(Archive &ar) {ar(text);};
};

// Register Bug
CEREAL_REGISTER_TYPE(Bug);
CEREAL_REGISTER_POLYMORPHIC_RELATION(Serializable, Bug);

int main()
{
    auto obj = make_shared<Bug>();

    std::ostringstream os;
    {
        cereal::BinaryOutputArchive oarchive(os);
        oarchive(dynamic_pointer_cast<Serializable>(obj));
    }

    shared_ptr<Serializable> obj2;
    std::istringstream is(os.str());
    {
        cereal::BinaryInputArchive iarchive(is);
        iarchive(obj2);
    }

    auto m = dynamic_pointer_cast<Bug>(obj2);

    return 0;
}

This code, when compiled with the latest nlohmann::json throws the following:

Trying to load an unregistered polymorphic type (Bug).
Make sure your type is registered with CEREAL_REGISTER_TYPE and that the archive you are using was included (and registered with CEREAL_REGISTER_ARCHIVE) prior to calling CEREAL_REGISTER_TYPE.
If your type is already registered and you still see this error, you may need to use CEREAL_REGISTER_DYNAMIC_INIT.

Curiously, though, it only happens when the name of the function in the Bug class is load, which is why I'm opening an issue with your library (it sseems to have special significance with your library).

Again, curiously, compiling against a 2.X version of nlohmann::json does not product this exception.

I noticed one change from nlohmann::json 2.X->3.X is the inclusion of the nlohmann::detail namespace. Could this be somehow conflicting with cereal::detail? I will also open an issue against nlohmann::json.

Still trying to figure out root cause

nlohmann::json issue: https://github.com/nlohmann/json/issues/1082

RPGillespie6 avatar May 07 '18 22:05 RPGillespie6

So this only triggers at runtime on that polymorphic check? Do you have any issues with non-polymoprhic code when using both libraries?

AzothAmmo avatar May 08 '18 05:05 AzothAmmo

It looks like I also have a problem with non-polymorphic code:

#include <json.hpp>

// #include <cereal/types/polymorphic.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/access.hpp>       //So we can make serialize private so developers aren't tempted to call it.
#include <cereal/types/string.hpp> //This is needed to serialize std::string. There are similar ones for the other std containers
#include <cereal/types/vector.hpp> //This is needed to serialize std::vector. There are similar ones for the other std containers

using namespace std;

class Bug
{
    public:
        std::string text;
        void load(const nlohmann::json & s) {};

    private:
        friend class cereal::access;
        template <class Archive> void serialize(Archive &ar) {ar(text);};
};


int main()
{
    Bug obj;
    std::ostringstream os;
    {
        cereal::BinaryOutputArchive oarchive(os);
        oarchive(obj);
    }

    Bug obj2;
    std::istringstream is(os.str());
    {
        cereal::BinaryInputArchive iarchive(is);
        iarchive(obj2);
    }

    return 0;
}

Has the following static assertion:

static assertion failed: cereal found more than one compatible input serialization function for the provided type and archive combination.

 Types must either have a serialize function, load/save pair, or load_minimal/save_minimal pair (you may not mix these). 
 Use specialization (see access.hpp) if you need to disambiguate between serialize vs load/save functions.  
 Note that serialization functions can be inherited which may lead to the aforementioned ambiguities. 
 In addition, you may not mix versioned with non-versioned serialization functions. 

However, if I either:

  1. Change the name from Bug::load to something else like Bug::loadJson
  2. Roll back nlohmann::json version

It magically works

RPGillespie6 avatar May 08 '18 12:05 RPGillespie6

I've also confirmed that the break occurs when moving from nlohmann::json version 2.0.10 -> 2.1.0. Version 2.1.0 introduces the nlohmann::detail namespace.

RPGillespie6 avatar May 08 '18 12:05 RPGillespie6

Update: I've confirmed it is definitely not a namespace collision. I renamed detail to an arbitrary name and I still see the same problem. Instead, it looks like somehow Cereal thinks the nlohmann::json type is a cereal::Archive such that when passed into a function named load it thinks I am registering a new load/save function. Unfortunately both libraries are extremely template heavy so it's hard to understand what's going on

RPGillespie6 avatar May 10 '18 15:05 RPGillespie6

Has a fix been found this problem? I'm seeing the same in my code (with the difference being that I am writing load and store functions to serialize a nlohmann::json into an archive). Cereal finds multiple serialization functions. I cannot find a load, store, or serialize function in the nlohmann-json code that could conflict, though...

mdorier avatar Jun 21 '21 14:06 mdorier