Question: How to adapt non-standard map implementation for serialization?

[ayeganov]

I am using this library https://github.com/greg7mdp/parallel-hashmap, and now have a need to serialize my data. I looked at the implementation of the std::unordered_map and followed that example to serialize parallel hashmap:

#pragma once

#include <cereal/cereal.hpp>
#include <parallel_hashmap/phmap.h>


namespace cereal
{
  namespace phmap_detail
  {
    template <class Archive, template<typename...> class Map, typename... Args, typename = typename Map<Args...>::mapped_type>
    void save( Archive & ar, Map<Args...> const & map )
    {
      ar( make_size_tag( static_cast<size_type>(map.size()) ) );

      for( const auto & i : map )
        ar( make_map_item(i.first, i.second) );
    }

    //! Loading for std-like pair associative containers
//    template <class Archive, typename Map>
    template <class Archive, template<typename...> class Map, typename... Args, typename = typename Map<Args...>::mapped_type>
    void load( Archive & ar, Map<Args...> & map )
    {
      size_type size;
      ar( make_size_tag( size ) );

      map.clear();

      auto hint = map.begin();
      for( size_t i = 0; i < size; ++i )
      {
        typename Map<Args...>::key_type key;
        typename Map<Args...>::mapped_type value;

        ar( make_map_item(key, value) );
        #ifdef CEREAL_OLDER_GCC
        hint = map.insert( hint, std::make_pair(std::move(key), std::move(value)) );
        #else // NOT CEREAL_OLDER_GCC
        hint = map.emplace_hint( hint, std::move( key ), std::move( value ) );
        #endif // NOT CEREAL_OLDER_GCC
      }
    }
  }


  //! Saving for phmap::node_hash_map
  template <class Archive, class K, class V>
  void CEREAL_SAVE_FUNCTION_NAME(Archive& ar, phmap::node_hash_map<K, V> const& map)
  {
    phmap_detail::save<Archive, phmap::node_hash_map, K, V>(ar, map);
  }

  //! Loading for phmap::node_hash_map
  template <class Archive, class K, class V>
  void CEREAL_LOAD_FUNCTION_NAME(Archive& ar, phmap::node_hash_map<K, V>& map)
  {
    phmap_detail::load<Archive, phmap::node_hash_map, K, V>(ar, map);
  }
}

I keep seeing this error:

/home/ainindza/.conan/data/cereal/1.3.0/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include/cereal/cereal.hpp:964:86: error: static assertion failed: cereal found more than one compatible input serialization function for the provided type and archive combination. 

 Types must either have a serialize function, load/save pair, or load_minimal/save_minimal pair (you may not mix these). 
 Use specialization (see access.hpp) if you need to disambiguate between serialize vs load/save functions.  
 Note that serialization functions can be inherited which may lead to the aforementioned ambiguities. 
 In addition, you may not mix versioned with non-versioned serialization functions. 

 
  964 |         static_assert(traits::detail::count_input_serializers<T, ArchiveType>::value < 2,
      |                                                                                ~~~~~~^~~

[ayeganov]

As a side note, when I use it directly in a unit test it works:

std::string archive_file = "/tmp/archive.bin";
{
    std::ofstream ofs(archive_file, std::ios::binary);
    cereal::BinaryOutputArchive ar(ofs);

    phmap::node_hash_map<core::PersonId, core::Person> test_map;
    test_map.try_emplace(12, 45, 100);
    ar(test_map);
}

Only when I use it with the rest of my application I am seeing the duplicates. I don't have any identical map definitions - the keys and values are always different, so I am having trouble understanding where the second identical function is coming from.

[patlecat]

I also wondered how well Cereal can serialize other custom map containers, there are a few out that perform much better than STL. What is needed to make them serializable?

[ayeganov]

@patlecat As a follow up - I have managed to make it work. The problem was actually stemming from the load function defined on the phmap::node_hash_map class, which was causing cereal to improperly count the number of methods of save/load. I ended up not needing my own implementation, and using the https://github.com/USCiLab/cereal/blob/master/include/cereal/types/concepts/pair_associative_container.hpp was enough to make everything work. I did need to define the PHMAP_NON_DETERMINISTIC macro, which removed the offending load function.

[dixyang]

Look #656 ,it sloved a problem like your , serials an non-standard type cv::point,it’s a type from temple, may make sense to you