[Draft] QDQ stripping transformation in OpenVINO EP

[sspintel]

trafficstars

[sspintel]

Thanks for the comments @skottmckay. I will address them. Meanwhile, a question. Is there an easy way to check if a given model is QDQ and only apply this transformation if that's the case (other than iterating through the whole graph and checking if a QDQ operator exists), as we don't want to build the graph from scratch for non-QDQ models.

[skottmckay]

Thanks for the comments @skottmckay. I will address them. Meanwhile, a question. Is there an easy way to check if a given model is QDQ and only apply this transformation if that's the case (other than iterating through the whole graph and checking if a QDQ operator exists), as we don't want to build the graph from scratch for non-QDQ models.

I'm not aware of any direct way.

If it isn't a QDQ model the number of nodes returned by GetAllNodeUnits should be equal to the number of nodes in the GraphViewer so you could potentially infer using that.

But calling GetAllNodeUnits is going to be more expensive than iterating and checking if any node's op_type is DQ/Q as a first step so it depends what you want to optimize for.

[sspintel]

Thanks for the comments @skottmckay. I will address them. Meanwhile, a question. Is there an easy way to check if a given model is QDQ and only apply this transformation if that's the case (other than iterating through the whole graph and checking if a QDQ operator exists), as we don't want to build the graph from scratch for non-QDQ models.

I'm not aware of any direct way.

If it isn't a QDQ model the number of nodes returned by GetAllNodeUnits should be equal to the number of nodes in the GraphViewer so you could potentially infer using that.

But calling GetAllNodeUnits is going to be more expensive than iterating and checking if any node's op_type is DQ/Q as a first step so it depends what you want to optimize for.

I'm iterating and checking if any node's op_type is DQ/Q instead and doing qdq stripping only it it's true

[sfatimar]

@jywu-msft

[jywu-msft]

/azp run Linux OpenVINO CI Pipeline

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[jywu-msft]

/azp run Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,onnxruntime-binary-size-checks-ci-pipeline,Big Models,Android CI Pipeline, Linux Android Emulator QNN CI Pipeline

[azure-pipelines[bot]]

Azure Pipelines successfully started running 10 pipeline(s).

[jywu-msft]

/azp run Linux OpenVINO CI Pipeline

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[jywu-msft]

/azp run Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,onnxruntime-binary-size-checks-ci-pipeline,Big Models,Android CI Pipeline, Linux Android Emulator QNN CI Pipeline

[azure-pipelines[bot]]

Azure Pipelines successfully started running 10 pipeline(s).

[jywu-msft]

/azp run Windows ARM64 QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline

[azure-pipelines[bot]]

Azure Pipelines successfully started running 10 pipeline(s).