Zero Zeng

@rajeevsrao ^ ^

should be "CustomGeluPluginDynamic"? see https://github.com/NVIDIA/TensorRT/blob/87f3394404ff9f9ec92c906cd4c39b5562aea42e/plugin/geluPlugin/geluPlugin.cpp#L38

If you turn on `--verbose` option in trtexec, can you see the log like ``` [08/26/2022-01:34:53] [I] TensorRT version: 8.4.1 [08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1...

Can you provide more details? how the log like and would be better if you can provide the onnx for reproduce.

`mark all` is not a good choice, it will break all tensorrt graph fusion and thus might change the inference result. I would suggest run it without `mark all` first...

Usually, this happened when your model has a dynamic input shape and a fixed reshape operation, can you check it first?

Check the onnx model first, e.g. run it with onnx runtime with a preset input shapes.

the problem here is simple, support you have a reshape layer, reshape a tensor to 2x6, it's has an input of axb, then axb must equal to 2x6=12

> I compare the output of q,k,attn of onnx and trt and print the max diff of each pair. q,k of them are the same, but attn are quite different....

I can't reproduce it using polygraphy, all output is matched: ``` [I] Accuracy Comparison | trt-runner-N0-08/22/22-15:50:44 vs. onnxrt-runner-N0-08/22/22-15:50:44 [I] Comparing Output: '72' (dtype=float32, shape=(8, 900, 32)) with '72' (dtype=float32, shape=(8,...