[Documentation] Attention Contrib Op Bias argument Not optional?

[TedThemistokleous]

Describe the documentation issue

Looking for some clarification on this one

Currently adding MIGraphX parser support for your Attention operator in with the intent to make it MIGraphX IR https://github.com/ROCm/AMDMIGraphX/pull/3816

OP in question https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#commicrosoftattention

I'm running into an issue when running onnx models generated that use the attention operator found in the Microsoft Contrib operator set segfault on CPU EP

It appears bias for the input projection is stated as an optional argument but I continually get segmentation faults through a run through OnnxRT CPU execution provider when I attempt to run a reduce model with minimum inputs (Input, Weights, Num_heads)

I've written a test in MIGraphX which was intended to test the min scale-dot attention head (set num_heads=1) and ensure the calculation maintains accuracy. I'm able to run things through MIGraphX but the issue only appears when things run through the EP

There's nothing that is clear in the attention_base.cc/.h files on inputs, and all I see that's required outside of input, and weights, and num_heads. Num_heads uses an ORT_ENFORCE() for the attribute, but is there something you set for required inputs in your operators?

All my other test seem to work correctly as long as bias is included in the input list. Is this an error or is there a change in documentation and the bias input is now required?

I can point you to the generated files to reproduce if this is a bug. We have a way to check accuracy on MIGraphX and compare between MIGraphX IR, CPU runs and other sources.

cc @tianleiwu

Page / URL

No response

[tianleiwu]

@TedThemistokleous, support of 'optional' of bias is added for T5 model in https://github.com/microsoft/onnxruntime/pull/14928. It is supported by CUDA provider. However, CPU provider still requires bias input.

[TedThemistokleous]

Oh ok, so are the contrib operators with respect to CUDA EP support or CPU EP? that isn't clear from the doc

[tianleiwu]

It's clear in operator spec.

I think CPU EP need slight modification to follow the operator spec, or update the error message for not-implemented feature to avoid confusing users. Let me add a work item to back log.