TensorRT Rename GELU Plugin have problems

Description

I use onnx-graphsurgeon to merge gelu ops, and rename Node in graph is "CustomGeluPlugin". I have also replaced plugin name from "CustomGeluPluginDynamic" to "CustomGeluPlugin". But when I use trt tools to generate trt engine from onnx model, trtexec --onnx=./myonnx.onnx --saveEngine=./myengine.trt --plugins=./libgelu.so --plugins=./liblaynorm.so --verbose Program always wait for searching this plugin: [I] [TRT] No importer registered for op: CustomGeluPlugin, Attempting to import as plugin. [I] [TRT] Searching for plugin: CustomGeluPlugin, plugin_version: 1, plugin_namespace:

But when I use the original plugin name, It's Ok. What's the problem?

Environment

TensorRT Version: 8.4.0.6 NVIDIA GPU: Tesla T4 NVIDIA Driver Version: 10.2 CUDA Version: 10.2 CUDNN Version: 8.3.2 Operating System: CentOS Baremetal or Container (if so, version): No

Aug 25 '22 03:08 oreo-lp

should be "CustomGeluPluginDynamic"? see https://github.com/NVIDIA/TensorRT/blob/87f3394404ff9f9ec92c906cd4c39b5562aea42e/plugin/geluPlugin/geluPlugin.cpp#L38

Aug 25 '22 15:08 zerollzeng

should be "CustomGeluPluginDynamic"? see

https://github.com/NVIDIA/TensorRT/blob/87f3394404ff9f9ec92c906cd4c39b5562aea42e/plugin/geluPlugin/geluPlugin.cpp#L38

if I want to change the plug-in name, what should I do?

Aug 25 '22 15:08 oreo-lp

should be "CustomGeluPluginDynamic"? see

https://github.com/NVIDIA/TensorRT/blob/87f3394404ff9f9ec92c906cd4c39b5562aea42e/plugin/geluPlugin/geluPlugin.cpp#L38

I wanna rename plugin, there is the steps: (1) I change GELU_PLUGIN_NAME from “CustomGeluPluiginDynamic ” to "CustomGeluPluigin". https://github.com/NVIDIA/TensorRT/blob/87f3394404ff9f9ec92c906cd4c39b5562aea42e/plugin/geluPlugin/geluPlugin.cpp#L38 (2) generate libgelu.so by CMakeLists.txt (3) merge GELU op in onnx by onnx_graphsurgeon: pluginNode = gs.Node("CustomGeluPluigin"， "geluplugin", inputs = [node.i().inputs[0]], outputs = node.outputs, attrs = {"type_id": 0, "bias}: 0) and save the merged onnx model. (4) use trtexec to generate engine model, but failed. trtexec --onnx=./myMergedOnnx.onnx --saveEngine=./myengine.trt --plugins=./libgelu.so --plugins=./liblaynorm.so --verbose Program always wait for searching this plugin: [I] [TRT] No importer registered for op: CustomGeluPlugin, Attempting to import as plugin. [I] [TRT] Searching for plugin: CustomGeluPlugin, plugin_version: 1, plugin_namespace:

If i wanna to rename the plugin, what should I do?

Aug 26 '22 01:08 oreo-lp

If you turn on --verbose option in trtexec, can you see the log like

[08/26/2022-01:34:53] [I] TensorRT version: 8.4.1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::CropAndResize version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::Proposal version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::Region_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::ScatterND version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::Split version 1
[08/26/2022-01:34:53] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1
[08/26/2022-01:37:19] [I] Loading supplied plugin library: xxx.so

which should contains your new plugin here.

Aug 26 '22 01:08 zerollzeng

我已经将--verbose打开了，但是在使用trtexec转模型的日志中并没有显示GeluPluginDynamicCreator注册了，但是它依然可以正常使用。而当我在进行推理的主程序中加入initLibNvInferPlugins()函数的时候，终端才会显示所有的自定义的plugin creator。我现在比较疑惑的问题是：如果我想修改plugin的名字，但是在使用trtexec将onnx模型转成trt模型的过程中，为什么会出这个creator找不到的问题？我只是修改了plugin的名字，为什么就不能注册了呢？（使用之前的CustomGeluPluginDynamic名字就可以注册）

Aug 26 '22 02:08 oreo-lp

@oreo-lp Did you change the op name in ONNX file? TRT is using ONNX's op name to search for plugins

Dec 02 '22 09:12 nvpohanh

Closing since no activity for more than 3 weeks, please reopen if you still have question, thanks!

Jan 10 '23 02:01 ttyio

TensorRT TensorRT copied to clipboard

Rename GELU Plugin have problems

Description

Environment

TensorRT
TensorRT copied to clipboard