tensorrt

tensorrt copied to clipboard

While trying to convert a RoBERTa model, `tf-trt` complains about `input_word_ids` not being in the graph. Nonetheless, if I run: ``` saved_model_cli show --dir {saved_model_dir} --all ``` it clearly shows...

dshahrokhian

## Description I tried to optimize a tensorflow 2+ simple classification model using tensorRT with as input a string as I want to serve with tensorflow serving with base64 input....

257kb

Error converting multilingual universal sentence encoder to TensorRT. Input 1 of node StatefulPartitionedCall was passed float from statefulpartitionedcall_args_1:0 incompatible with expected resource.

1

I'm getting below error while converting multilingual universal sentence encoder(https://tfhub.dev/google/universal-sentence-encoder-multilingual/3) to TensorRT using `TrtGraphConverterV2` `InvalidArgumentError: Input 1 of node StatefulPartitionedCall was passed float from statefulpartitionedcall_args_1:0 incompatible with expected resource.` Below...

sebastinajay

uff.model.exceptions.UffException: Const node conversion requested, but node is not Const

3

Traceback (most recent call last): File "/home/ubuntu/anaconda3/bin/convert-to-uff", line 10, in sys.exit(main()) File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/uff/bin/convert_to_uff.py", line 92, in main debug_mode=args.debug File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/uff/converters/tensorflow/conversion_helpers.py", line 229, in from_tensorflow_frozen_model return from_tensorflow(graphdef, output_nodes, preprocessor, **kwargs) File...

gftd

converter build with converter from tf.estimator pb is not working

alanpurple

**model loading** `def load_tf_saved_model(input_saved_model_dir): if not tf.compat.v1.saved_model.contains_saved_model(input_saved_model_dir): print("Please give valid saved model folder %s"%input_saved_model_dir) print('TF Loading saved model {}...'.format(input_saved_model_dir)) if tf.__version__ == '1.15.0': saved_model_loaded = tf.saved_model.load_v2(input_saved_model_dir, tags=[tag_constants.SERVING]) infer = saved_model_loaded.signatures[...

AhmadShaik

Run environment: - docker version: nvcr.io/nvidia/tensorflow:20.01-tf2-py3 with tensorflow version 2.0 and TensorRT 7.0 - Validation TFRecord files: [https://github.com/tensorflow/models/blob/master/research/slim/datasets/build_imagenet_data.py](url) - Model: TF2.0 Resnet_50 hub model [https://tfhub.dev/tensorflow/resnet_50/classification/1](url) - preprocess_method: vgg Why is...

xlvy

used docker conteiner: **nvcr.io/nvidia/tensorflow:19.10-py3** used script: **modfificated object_detection.py** for test created engine used trtexec in docker conteiner: ``` &&&& RUNNING TensorRT.trtexec # ./trtexec --loadEngine=/home/user/disc_brand/yolov3_INT8_1_segment0.trtengine --int8 [09/15/2020-17:13:02] [I] === Model Options...

daliel

image_tensor = graph.get_tensor_by_name('wpf:0') calib_graph = trt.create_inference_graph( input_graph_def=graph_def, outputs=output_names, max_batch_size=1, minimum_segment_size=4, is_dynamic_op=True, maximum_cached_engines=3, max_workspace_size_bytes=trt.DEFAULT_TRT_MAX_WORKSPACE_SIZE_BYTES, precision_mode='INT8' ) a,b,c,d = tf.import_graph_def(calib_graph, input_map={"wpf": image_tensor}, return_elements=output_names, name="") a, b, c, d = sess.run([a,b,c,d], feed_dict={image_tensor: [cv2.resize(cv2.imread("../images/1.jpg"),(608,600))]})...

wpf19911118

Native TensorFlow is >6 times faster than TensorRT

2

## Description My neural network runs much faster with native TensorFlow compared to a TensorRT optimized model: Images per second with native TF: 4.785973 Images per second with TRT: 0.712366...

olesalscheider