[Feature Request: c#] Provide Onnxruntime related Nugets consisting of contributed Operators as shared library

[GeorgeS2019]

Describe the feature request

• We now have improved documentation on how to consume (through registration) custom operators which is relevant to c# users
• There is a new effort in refactoring the c and c++ API for dealing with custom operators, which I hope will be relevant to this feature request.

It is time we consider how to provide different Onnxruntime related NuGets which consist of shared libraries that expose contributed operators similar in format to custom_op_library.dll and therefore accessible to c# users

Describe scenario use cases

Reference

use case 1

• [ ] Evaluate the feasibility to use TorchSharp to integrate custom operators and export to ONNX.
• [ ] Use ML.NET to register the integrated custom operators used in the exported ONNX model before consuming the ONNX model

use case 2

• [ ] Instead of integrating custom operators as stated in use case 1, integrate existing contributed operators distributed by ONNXRUNTIME
• [ ] Use ML.NET to register the integrated contributed operators used in the exported ONNX model before consuming the ONNX model

[snnn]

Contrib Operators are not custom kernels. Contrib operators are schemas that are statically linked into onnxruntime.dll. Normally their implementations are also linked to the core runtime in such a way. Custom kernels are implementations of some operator schemas in separated DLLs. You may implement a contrib operator schema in such a way, but normally we think Contrib ops and contrib ops are not related as their kernel interfaces are very different.

[GeorgeS2019]

@snnn Still learning the difference :-)

In you view, how to make Contrib operators accessible like the custom operator and make this available as separate DLLs?

[snnn]

If they are already in the main DLL, then they are. If not, then no way.

[GeorgeS2019]

@snnn

Please kindly share where do I find contributed operators that are already part of the main DLL and those which are not.

I would like to learn how to tract those which are in and those which are not.

Question

What is the benefit of having the listed contributed operators and yet not including them in the Main Dll?

[natke]

@GeorgeS2019 Can you email me at [email protected] and I can share this information with you

[GeorgeS2019]

Examples of pre/post CustomOps as a shared library

Through further offline communication, it seems the Onnx team has started to create packages for this library

Deep Vision: pre/post processing CustomOps

• [x] OpenCV Image Decoder & SuperResolution CustomOps

Deep NLP: pre/post processing CustomOps

Reference: [Feature Request] Classify Contributed Ops The priority will be those Ops that are not related to tokenization BUT perform post-processing operations such as

nlp

com.microsoft.nlp

• com.microsoft.nlp.BeamSearch
• com.microsoft.nlp.GreedySearch

[faxu]

The contrib ops are listed here: https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md These are all included and packaged as part of the release binaries.

[yuslepukhin]

I do not think we should expose them as a separate entity. This is not a supportable pubic interface. We use it for optimization purposes and some other internal needs. It is important that we are able to innovate freely when outside the standard boundaries.

[GeorgeS2019]

able to innovate freely when outside the standard boundaries. @yuslepukhin could u elaborate.

something like this? : OpenCV Image Decoder & SuperResolution CustomOps