how to convert Transformer model with tensorRT ops

[lyzKF]

Hi, i am trying to convert Transformer model with tensorRT ops, the model is trained based on Opennmt-tf. here is my scripts:

CUDA_VISIBLE_DEVICES=0 saved_model_cli convert --dir ./averaged-ende-export500k-v2 --tag_set serve --output_dir ./trt_export/ tensorrt

errors: 2022-07-28 07:31:39.262262: W tensorflow/core/framework/op_kernel.cc:1692] OP_REQUIRES failed at trt_engine_resource_ops.cc:191 : Not found: Container TF-TRT does not exist. (Could not find resource: TF-TRT/TRTEngineOp_0_49) INFO:tensorflow:Could not find TRTEngineOp_0_49 in TF-TRT cache. This can happen if build() is not called, which means TensorRT engines will be built and cached at runtime. Traceback (most recent call last): File "/usr/local/bin/saved_model_cli", line 8, in sys.exit(main()) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/tools/saved_model_cli.py", line 1192, in main args.func(args) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/tools/saved_model_cli.py", line 794, in convert_with_tensorrt converter.save(output_saved_model_dir=args.output_dir) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/compiler/tensorrt/trt_convert.py", line 1294, in save save.save(self._saved_model, output_saved_model_dir, signatures) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/save.py", line 1193, in save save_and_return_nodes(obj, export_dir, signatures, options) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/save.py", line 1228, in save_and_return_nodes _build_meta_graph(obj, signatures, options, meta_graph_def)) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/save.py", line 1399, in _build_meta_graph return _build_meta_graph_impl(obj, signatures, options, meta_graph_def) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/save.py", line 1339, in _build_meta_graph_impl signature_serialization.canonicalize_signatures(signatures)) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/signature_serialization.py", line 180, in canonicalize_signatures final_concrete = signature_wrapper._get_concrete_function_garbage_collected( # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 1213, in _get_concrete_function_garbage_collected self._initialize(args, kwargs, add_initializers_to=initializers) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 759, in _initialize self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 3066, in _get_concrete_function_internal_garbage_collected graph_function, _ = self._maybe_define_function(args, kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 3463, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 3298, in _create_graph_function func_graph_module.func_graph_from_py_func( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py", line 1007, in func_graph_from_py_func func_outputs = python_func(*func_args, **func_kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 668, in wrapped_fn out = weak_wrapped_fn().wrapped(*args, **kwds) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py", line 994, in wrapper raise e.ag_error_metadata.to_exception(e) TypeError: in user code:

/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/signature_serialization.py:157 signature_wrapper  *
    structured_outputs = signature_function(**kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:1707 __call__  **
    return self._call_impl(args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/wrap_function.py:246 _call_impl
    return super(WrappedFunction, self)._call_impl(
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:1725 _call_impl
    return self._call_with_flat_signature(args, kwargs, cancellation_manager)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py:1756 _call_with_flat_signature
    raise TypeError("{} missing required arguments: {}".format(

TypeError: pruned(length, tokens, unknown, unknown_261) missing required arguments: unknown, unknown_261

looking forward to your reply, thank you