cv::Mat + string serialization

[MyraBaba]

Hi,

I would love to use msgpack in our projects.

Can we directly use cvmat to serialize ?

How I can pack below struct in msgpack to send through zeromq?

struct Content{
  cv:Mat image;
  std::string  msg;

}

[nobu-k]

This example describes how to serialize a struct with msgpack using MSGPACK_DEFINE. However, in your code above, cv::Mat cannot directly be used with MSGPACK_DEFINE, and you need to implement what MSGPACK_DEFINE does manually.

MSGPACK_DEFINE implements following member functions in a struct or a class:

template <typename Packer>
void msgpack_pack(Packer& pk) const; // serialize
void msgpack_unpack(msgpack::object const& o); // deserialize

You need to implement each member function in Content. Packer's source code is here. It basically provides pack or pack_{type} functions to serialize primitive types. Note that you have to serialize your struct into an array or a map so that it can be deserialized by msgpack_unpack via one msgpack::object.

I'm not sure how cv::Mat is commonly serialized, but this article could be your help.

You can also implement a serialization and deserialization feature for cv::Mat in a non-intrusive way so that cv::Mat can be used with MSGPACK_DEFINE directly.

[pfeatherstone]

i have the following adaptors:

#include <msgpack/versioning.hpp>
#include <msgpack/adaptor/adaptor_base.hpp>
#include <msgpack/meta.hpp>
#include <msgpack/object.hpp>
#include <opencv2/core.hpp>

namespace msgpack {

/// @cond
MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS) {
/// @endcond

namespace adaptor {

static const std::string OPENCV_MAT_KEYS[] = {
    "rows",
    "cols",
    "type",
    "data"
};

template<>
struct pack<cv::Mat> {
    template <typename Stream>
    packer<Stream>& operator()(msgpack::packer<Stream>& o, cv::Mat const& v) const {
        const int nbytes = v.total() * v.elemSize();
        o.pack_map(4);
        o.pack(OPENCV_MAT_KEYS[0]);
        o.pack(v.rows);
        o.pack(OPENCV_MAT_KEYS[1]);
        o.pack(v.cols);
        o.pack(OPENCV_MAT_KEYS[2]);
        o.pack(v.type());
        o.pack(OPENCV_MAT_KEYS[3]);
        o.pack_array(nbytes);
        for (int i = 0 ; i < nbytes ; i++)
            o.pack(v.data[i]);
        return o;
    }
};

template <>
struct convert<cv::Mat> 
{
    msgpack::object const& operator()(msgpack::object const& o, cv::Mat& v) const 
    {
        if(o.type != msgpack::type::MAP)
            throw msgpack::type_error();
        if(o.via.map.size != 4)
            throw msgpack::type_error();

        std::string keys[4];
        for (int key_index = 0 ; key_index < 4 ; key_index++)
        {
            o.via.map.ptr[key_index].key.convert(keys[key_index]);
            if (keys[key_index] != OPENCV_MAT_KEYS[key_index])
                throw msgpack::type_error();
        }

        if (o.via.map.ptr[3].val.type != msgpack::type::ARRAY)
            throw msgpack::type_error();

        int rows, cols, type;
        o.via.map.ptr[0].val.convert(rows);
        o.via.map.ptr[1].val.convert(cols);
        o.via.map.ptr[2].val.convert(type);

        v = cv::Mat(rows, cols, type);
        const uint32_t nbytes = v.total() * v.elemSize();

        if (o.via.map.ptr[3].val.via.array.size != nbytes)
            throw msgpack::type_error();

        for (uint32_t i = 0 ; i < nbytes ; i++)
            o.via.map.ptr[3].val.via.array.ptr[i].convert(v.data[i]);
        return o;
    }
};

} // namespace adaptor

/// @cond
}  // MSGPACK_API_VERSION_NAMESPACE(v1)
/// @endcond

}  // namespace msgpack

However it is super bloated. i have an image that is 768x576. Sot that's 1327104 bytes just for the pixels. Using the above adaptors, when serialised, i get a buffer of size 2107461 bytes. I know there is some overhead with the headers, but that's a lot of overhead. Can anyone spot something stupid in the code above?

[pfeatherstone]

Ok, need to use BIN instead of array for the pixels. Now i get 1327137 serialised bytes for 1327104 pixels. that's better. Here is the code:

#include <msgpack/versioning.hpp>
#include <msgpack/adaptor/adaptor_base.hpp>
#include <msgpack/meta.hpp>
#include <msgpack/object.hpp>
#include <opencv2/core.hpp>

namespace msgpack {

/// @cond
MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS) {
/// @endcond

namespace adaptor {

static const std::string OPENCV_MAT_KEYS[] = {
    "rows",
    "cols",
    "type",
    "data"
};

template<>
struct pack<cv::Mat> {
    template <typename Stream>
    packer<Stream>& operator()(msgpack::packer<Stream>& o, cv::Mat const& v) const {
        const int nbytes = v.total() * v.elemSize();
        o.pack_map(4);
        o.pack(OPENCV_MAT_KEYS[0]);
        o.pack(v.rows);
        o.pack(OPENCV_MAT_KEYS[1]);
        o.pack(v.cols);
        o.pack(OPENCV_MAT_KEYS[2]);
        o.pack(v.type());
        o.pack(OPENCV_MAT_KEYS[3]);
        o.pack_bin(nbytes);
        o.pack_bin_body((const char*)v.data, nbytes);
        return o;
    }
};

template <>
struct convert<cv::Mat> 
{
    msgpack::object const& operator()(msgpack::object const& o, cv::Mat& v) const 
    {
        if(o.type != msgpack::type::MAP)
            throw msgpack::type_error();
        if(o.via.map.size != 4)
            throw msgpack::type_error();

        std::string keys[4];
        for (int key_index = 0 ; key_index < 4 ; key_index++)
        {
            o.via.map.ptr[key_index].key.convert(keys[key_index]);
            if (keys[key_index] != OPENCV_MAT_KEYS[key_index])
                throw msgpack::type_error();
        }

        if (o.via.map.ptr[3].val.type != msgpack::type::BIN)
            throw msgpack::type_error();

        int rows, cols, type;
        o.via.map.ptr[0].val.convert(rows);
        o.via.map.ptr[1].val.convert(cols);
        o.via.map.ptr[2].val.convert(type);

        v = cv::Mat(rows, cols, type);
        const uint32_t nbytes = v.total() * v.elemSize();

        if (o.via.map.ptr[3].val.via.bin.size != nbytes)
            throw msgpack::type_error();

        memcpy(v.data, o.via.map.ptr[3].val.via.bin.ptr, nbytes);
        return o;
    }
};

} // namespace adaptor

/// @cond
}  // MSGPACK_API_VERSION_NAMESPACE(v1)
/// @endcond

}  // namespace msgpack

I can't be bothered to submit a PR. So feel free to do so.