Tamas Bela Feher

To avoid long startup time please call `converter.build()` before you save the model, see the [example here](https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#worflow-with-savedmodel). Otherwise only a placeholder for the TRT engine is saved in the graph,...

Thanks @devalexqt for the update. TF-TRT would create a new engine every time it sees input shape which it cannot handle with the existing engine. For example, if you create...

/merge

Linking discussion about template parameters for filters: https://github.com/rapidsai/raft/pull/2212#issuecomment-1979439358, and related task - [ ] avoid recompiling `ivf:pq::search` kernels when index type changes

The main question here what is the best mechanism for the user to opt in/out of 1xTF32 computation. @vinaydes is working on the same question related to #1892. Let's wait...

Thanks @DEKHTIARJonathan for the PR! Before we merge this, the notebooks should be updated to explain how to run TF-TRT in Colab. Here is some code to get started: ```python...

To debug the problem, I have copied the code from trt_convert.py here: ```python func, model = get_func_from_saved_model(bert_saved_model_path) # Create frozen func from tensorflow.python.framework import convert_to_constants frozen_func = convert_to_constants.convert_variables_to_constants_v2(func) # Prepare...

@bixia1 While the conversion of TF Hub Bert Large models fail, there are other versions of the BERT large models that can be converted with TF-TRT. This includes the [NGC...

TF-TRT currently does not convert the LSTM op (see [supported ops](https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#supported-ops)). TensorRT itself supports LSTMs through its [RNN layer](https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/infer/Graph/Layers.html#irnnv2layer), but TF-TRT does not convert the TF LSTM into TensorRT LSTM....

No, we still do not have support for LSTM in TF-TRT. Alternative to TF-TRT, you could try if the [tf2onnx -> onnx-tensorrt route](https://developer.nvidia.com/blog/speeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt/) works for you.