submemory

[affranchi]

Hi,

I'm trying to reshape with submemory for binary operators. Source 1 is 2x16 matrix and Source 2 is 16 size vector and destination is 16 size vector. When I do binary_add and binary_mul with src1_desc.submemory_desc({1,16},{0,0}).reshape({16}), src2_desc, dst_desc is working. However, src1_desc.submemory_desc({1,16},{1,0}).reshape({16}), src2_desc, dst_desc is not working. It's working if I put reshaped src1 as second source and src2 as first source. Is it expected behavior? I'm just wondering why the first scenario - (0,0) offset submemory - is working but the second scenario is not working - (1,0) offset submemory -.

The below code is working - first scenario -:

    auto src1_desc = dnnl::memory::desc({2,16}, dnnl::memory::data_type::f32, dnnl::memory::format_tag::ab);
    auto src2_desc = dnnl::memory::desc({16}, dnnl::memory::data_type::f32, dnnl::memory::format_tag::a);
    auto dst_desc = dnnl::memory::desc({16}, dnnl::memory::data_type::f32, dnnl::memory::format_tag::a);

    auto engine = dnnl::engine(dnnl::engine::kind::cpu, 0);
    auto src1_mem = dnnl::memory(src1_desc, engine);
    auto src2_mem = dnnl::memory(src2_desc, engine);
    auto dst_mem = dnnl::memory(dst_desc, engine);

    float* src1_data = static_cast<float*>(src1_mem.get_data_handle());
    float* src2_data = static_cast<float*>(src2_mem.get_data_handle());
    float* dst_data = static_cast<float*>(dst_mem.get_data_handle());
 
    auto binary_d
        = dnnl::binary::desc(dnnl::algorithm::binary_add, src1_desc.submemory_desc({1,16},{0,0}).reshape({16}), src2_desc, dst_desc);
    auto binary_pd = dnnl::binary::primitive_desc(binary_d, engine);
    auto binary_prim = dnnl::binary(binary_pd);
    std::unordered_map<int, dnnl::memory> binary_args;
    binary_args.insert({DNNL_ARG_SRC_0, src1_mem});
    binary_args.insert({DNNL_ARG_SRC_1, src2_mem});
    binary_args.insert({DNNL_ARG_DST, dst_mem});
    auto engine_stream = dnnl::stream(engine);
    binary_prim.execute(engine_stream, binary_args);

Not working (the second scenario - submemory with {1,0} offset - "error: could not create a descriptor for a binary operation primitive" :

    auto src1_desc = dnnl::memory::desc({2,16}, dnnl::memory::data_type::f32, dnnl::memory::format_tag::ab);
    auto src2_desc = dnnl::memory::desc({16}, dnnl::memory::data_type::f32, dnnl::memory::format_tag::a);
    auto dst_desc = dnnl::memory::desc({16}, dnnl::memory::data_type::f32, dnnl::memory::format_tag::a);

    auto engine = dnnl::engine(dnnl::engine::kind::cpu, 0);
    auto src1_mem = dnnl::memory(src1_desc, engine);
    auto src2_mem = dnnl::memory(src2_desc, engine);
    auto dst_mem = dnnl::memory(dst_desc, engine);

    float* src1_data = static_cast<float*>(src1_mem.get_data_handle());
    float* src2_data = static_cast<float*>(src2_mem.get_data_handle());
    float* dst_data = static_cast<float*>(dst_mem.get_data_handle());
 
    auto binary_d
        = dnnl::binary::desc(dnnl::algorithm::binary_add, src1_desc.submemory_desc({1,16},{1,0}).reshape({16}), src2_desc, dst_desc);
    auto binary_pd = dnnl::binary::primitive_desc(binary_d, engine);
    auto binary_prim = dnnl::binary(binary_pd);
    std::unordered_map<int, dnnl::memory> binary_args;
    binary_args.insert({DNNL_ARG_SRC_0, src1_mem});
    binary_args.insert({DNNL_ARG_SRC_1, src2_mem});
    binary_args.insert({DNNL_ARG_DST, dst_mem});
    auto engine_stream = dnnl::stream(engine);
    binary_prim.execute(engine_stream, binary_args);

Working with the second scenario offset but put the one in the second source:

    auto src1_desc = dnnl::memory::desc({2,16}, dnnl::memory::data_type::f32, dnnl::memory::format_tag::ab);
    auto src2_desc = dnnl::memory::desc({16}, dnnl::memory::data_type::f32, dnnl::memory::format_tag::a);
    auto dst_desc = dnnl::memory::desc({16}, dnnl::memory::data_type::f32, dnnl::memory::format_tag::a);

    auto engine = dnnl::engine(dnnl::engine::kind::cpu, 0);
    auto src1_mem = dnnl::memory(src1_desc, engine);
    auto src2_mem = dnnl::memory(src2_desc, engine);
    auto dst_mem = dnnl::memory(dst_desc, engine);

    float* src1_data = static_cast<float*>(src1_mem.get_data_handle());
    float* src2_data = static_cast<float*>(src2_mem.get_data_handle());
    float* dst_data = static_cast<float*>(dst_mem.get_data_handle());
 
    auto binary_d
        = dnnl::binary::desc(dnnl::algorithm::binary_add, src2_desc, src1_desc.submemory_desc({1,16},{1,0}).reshape({16}), dst_desc);
    auto binary_pd = dnnl::binary::primitive_desc(binary_d, engine);
    auto binary_prim = dnnl::binary(binary_pd);
    std::unordered_map<int, dnnl::memory> binary_args;
    binary_args.insert({DNNL_ARG_SRC_0, src2_mem});
    binary_args.insert({DNNL_ARG_SRC_1, src1_mem});
    binary_args.insert({DNNL_ARG_DST, dst_mem});
    auto engine_stream = dnnl::stream(engine);
    binary_prim.execute(engine_stream, binary_args);

Thanks, Kenny

[dzarukin]

Hi @affranchi,

Thanks for sharing your report with us.

In scenario 2 binary::desc fails due to the fact src0_desc and dst_desc are not the same: https://github.com/oneapi-src/oneDNN/blob/master/src/common/binary.cpp#L63. The desc comparsion includes offset0 fields too, which is different for reshaped over submemoried src0_desc (equals to 16) versus dst_desc (equals to 0). This requirement came from the fact binary support in-place operation.

We tend to agree that this is strict limitation for out-of-place case, so we will investigate if simply removing the check and make binary work with submemory will do the trick. Once confirmed, we'll share a patch with you to fix a problem.

Will it work for you?

Thanks.

[affranchi]

Thank you for your answer, @dzarukin . It's very helpful to understand behavior. It would be great if binary ops work with submemory desc.

I'm not sure if you want to leave it open for tracking or not. So, I'll leave it this thread open. Please, feel free to close it.

Thanks.