rfc: add proposal on making opdesc and md opaque

[densamoilov]

Link to the document.

@TaoLv, @Jianhui-Li @jgong5 @vladimir-paramuzov @dmitry-gorokhov @yiqianglee

[hyunback]

Hi @densamoilov, I quickly reviewed opdesc and some question. Making opdesc seems it can't update or changes format_desc(blocking-inner blk, idx...). If ture, Is there any way to create dnnl::memory::desc with unsupported tag? e.g) aBcd2b, ABcd16a2b, ABcde16a4b... Current, openVINO creates dnnl::memory::desc with unspported tag from parsing string likes "aBcd2b" I can't find proper API with not defined TAG.. Is there a plan to support NOT defined TAG?

[densamoilov]

Hi @hyunback, this is a good question. You are right that there is no an API that would allow users to create a memory descriptor for unsupported tags. I’ve been waiting for feedback from OpenVINO on this. Can you please clarify when and why do you need to create a memory descriptor for unsupported tags? Given that oneDNN doesn’t have optimised kernels for such tags and only slow reference implementations would be used it’s not actually clear to me why such a capability could be useful other than for interoperability purpose.

[hyunback]

Yes, basically using any-tag can clear this issues, But OpenVINO doesn't use format_tag "any" NOW. That means we don't query it, just add proper format to run jit:ir in the code. Of course we have a plan to use format_tag "any", but it takes some times many code changes and verification too. So I want to know the schedule, when is expected target milestone for md opaque? BTW unsupported TAG is familiar things, For example oneDNNv2.7 requests aBcd2b, ABcd16a2b, ABcd16a4b format in shallow( feature depth <= 8) convolution input format to run jit:ir kernel and these formats are not defined in oneDNN tag enumeration now.

[densamoilov]

Added an option to remove operation descriptor from the API.

[densamoilov]

@hyunback, after an offline discussion we concluded that adding missing format tags will address your concern. Is my understanding correct?

[densamoilov]

@yiqianglee, @jgong5, @vladimir-paramuzov, @dmitry-gorokhov, we need your feedback on this proposal. Currently, we are leaning towards making memory descriptor opaque and removing operation descriptors from API. Please let us know whether the changes are acceptable to you.

[dmitry-gorokhov]

@densamoilov In general the proposal looks good to me. Regarding opaque memory descriptor: acceptable options for OpenVINO are 1 and 2a. Option 2 is not acceptable at the same time since some implementation details are highly utilized by OpenVINO CPU and GPU integrations.

[jgong5]

@yiqianglee, @jgong5, @vladimir-paramuzov, @dmitry-gorokhov, we need your feedback on this proposal. Currently, we are leaning towards making memory descriptor opaque and removing operation descriptors from API. Please let us know whether the changes are acceptable to you.

For PyTorch, we are ok with option 2a for memory descriptor and option2 for primitive descriptor. cc @XiaobingSuper

[yiqianglee]

@yiqianglee, @jgong5, @vladimir-paramuzov, @dmitry-gorokhov, we need your feedback on this proposal. Currently, we are leaning towards making memory descriptor opaque and removing operation descriptors from API. Please let us know whether the changes are acceptable to you.

@densamoilov , similar as PyTorch, ITEX team is fine with option 2a for memory descriptor, and option 2 for primitive descriptor.

BTW, for option 2a for memory descriptor, what's the plan for sparse memory?

[densamoilov]

BTW, for option 2a for memory descriptor, what's the plan for sparse memory?

I think the general idea is to introduce a notion of dense and sparse memory descriptor. In the future we will have two sets of API for creating memory descriptor:

1. Dense - the existing API will be used to create a dense memory descriptor with the given strides or format tag
2. Sparse - a new API will be used to create a sparse memory descriptor

There will also be an API to query memory descriptor kind (dense or sparse). Dense and sparse memory descriptor will have separate semantics (e.g. format kind is not applicable to sparse memory descriptors).

I put some high-level details in this paragraph.

[jondea]

Hi @densamoilov, I've had a look through the RFC, I understand that the main focus of this is to change the user API, but will we still have low level control of memory descriptor from within primitives? For example, we manually access the strides in https://github.com/oneapi-src/oneDNN/blob/aabd5652eab2c14b0324cc1e162d3eb1b158b597/src/cpu/aarch64/acl_utils.cpp#L193

But we may also want to going further than this. If a primitive receives a memory descriptor with format_kind::any (or the new equivalent), then the primitive may want to manually set the strides, inner_blks, inner_idxs etc rather than initialize the descriptor using a format_tag. Would we still be able to do this?

[densamoilov]

Hi @jondea, yes, inside the library everything stays as it is now.

[densamoilov]

The changes have been implemented and merged to master (last commit: 3bb87e686a72e70580f2a661cbf7ef2a6dd8029a).