[ROCm] NGramRepeatBlock, LongformerAttention and DecoderAttention Ops

[xinyazhang]

Description: Add NGramRepeatBlock, LongformerAttention and DecoderAttention Op to ROCm and/or enable their tests

Motivation and Context

• Why is this change required? What problem does it solve?
  • Add and test NGramRepeatBlock, LongformerAttention and DecoderAttention Ops.

[azure-pipelines[bot]]

Pull request contains merge conflicts.

[azure-pipelines[bot]]

Pull request contains merge conflicts.

[ytaous]

can u pls resolve conflicts first? thx

[xinyazhang]

can u pls resolve conflicts first? thx

Updated @ytaous

[ytaous]

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux Nuphar CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline

[ytaous]

/azp run Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows WebAssembly CI Pipeline, orttraining-amd-gpu-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed, onnxruntime-python-checks-ci-pipeline

[azure-pipelines[bot]]

Azure Pipelines successfully started running 9 pipeline(s).

[azure-pipelines[bot]]

Azure Pipelines successfully started running 8 pipeline(s).

[ytaous]

pls fix lint error

[xinyazhang]

pls fix lint error

A few remaining lint problems:

1. s = s.replace(...) in python, which is common practice.
2. long long int usage in onnxruntime/core/providers/rocm/shared_inc/fpgeneric.h, which is the practice of cuda version.

[ytaous]

/azp run Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows WebAssembly CI Pipeline, orttraining-amd-gpu-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed, onnxruntime-python-checks-ci-pipeline

[ytaous]

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux Nuphar CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline

[azure-pipelines[bot]]

Azure Pipelines successfully started running 8 pipeline(s).

[azure-pipelines[bot]]

Azure Pipelines successfully started running 9 pipeline(s).

[xinyazhang]

I think the C++ lint error is a false alert. Those lines do not exceed the 120-character limit The Python is the conventional s = s.replace(...) practice.

[ytaous]

Hi any update on this? No hurry, we can try to close the others before this one, thx.

[xinyazhang]

Hi any update on this? No hurry, we can try to close the others before this one, thx.

Working on PR #11968 and #11972 now. Will go back to this one after pushing necessary changes.

[xinyazhang]

$ (cd ./branch_build/$(git branch --show-current)/RelWithDebInfo/; ./onnxruntime_test_all --gtest_filter='NGramRepeatBlockTest.*:LongformerAttentionTest.*:DecoderAttentionTest.*:')
...
[----------] Global test environment tear-down
[==========] 13 tests from 3 test suites ran. (4578 ms total)
[  PASSED  ] 13 tests.

GPU Utilization confirmed with AMD_LOG_LEVEL=3.

Full log files: pool_v2.log pool_v2_gpu.log

[ytaous]

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux Nuphar CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline

[ytaous]

/azp run Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows WebAssembly CI Pipeline, orttraining-amd-gpu-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed, onnxruntime-python-checks-ci-pipeline

[azure-pipelines[bot]]

Azure Pipelines successfully started running 9 pipeline(s).

[azure-pipelines[bot]]

Azure Pipelines successfully started running 8 pipeline(s).

[ytaous]

fyi - https://github.com/microsoft/onnxruntime/pull/12435

[ytaous]

fyi - #12435

it's merged, can u please update it as needed? thx

[ytaous]

/azp run Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows WebAssembly CI Pipeline, orttraining-amd-gpu-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed, onnxruntime-python-checks-ci-pipeline

[ytaous]

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux Nuphar CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline

[azure-pipelines[bot]]

Azure Pipelines successfully started running 8 pipeline(s).

[azure-pipelines[bot]]

Azure Pipelines successfully started running 9 pipeline(s).

[ytaous]

heads up - https://github.com/microsoft/onnxruntime/pull/12448

[ytaous]

@iK1D - pls also take a look, thx