tensorrt
tensorrt copied to clipboard
I can't create a work plan with int8 precision
used docker conteiner: nvcr.io/nvidia/tensorflow:19.10-py3 used script: modfificated object_detection.py for test created engine used trtexec in docker conteiner:
&&&& RUNNING TensorRT.trtexec # ./trtexec --loadEngine=/home/user/disc_brand/yolov3_INT8_1_segment0.trtengine --int8
[09/15/2020-17:13:02] [I] === Model Options ===
[09/15/2020-17:13:02] [I] Format: *
[09/15/2020-17:13:02] [I] Model:
[09/15/2020-17:13:02] [I] Output:
[09/15/2020-17:13:02] [I] === Build Options ===
[09/15/2020-17:13:02] [I] Max batch: 1
[09/15/2020-17:13:02] [I] Workspace: 16 MB
[09/15/2020-17:13:02] [I] minTiming: 1
[09/15/2020-17:13:02] [I] avgTiming: 8
[09/15/2020-17:13:02] [I] Precision: INT8
[09/15/2020-17:13:02] [I] Calibration: Dynamic
[09/15/2020-17:13:02] [I] Safe mode: Disabled
[09/15/2020-17:13:02] [I] Save engine:
[09/15/2020-17:13:02] [I] Load engine: /home/user/disc_brand/yolov3_INT8_1_segment0.trtengine
[09/15/2020-17:13:02] [I] Inputs format: fp32:CHW
[09/15/2020-17:13:02] [I] Outputs format: fp32:CHW
[09/15/2020-17:13:02] [I] Input build shapes: model
[09/15/2020-17:13:02] [I] === System Options ===
[09/15/2020-17:13:02] [I] Device: 0
[09/15/2020-17:13:02] [I] DLACore:
[09/15/2020-17:13:02] [I] Plugins:
[09/15/2020-17:13:02] [I] === Inference Options ===
[09/15/2020-17:13:02] [I] Batch: 1
[09/15/2020-17:13:02] [I] Iterations: 10 (200 ms warm up)
[09/15/2020-17:13:02] [I] Duration: 10s
[09/15/2020-17:13:02] [I] Sleep time: 0ms
[09/15/2020-17:13:02] [I] Streams: 1
[09/15/2020-17:13:02] [I] Spin-wait: Disabled
[09/15/2020-17:13:02] [I] Multithreading: Enabled
[09/15/2020-17:13:02] [I] CUDA Graph: Disabled
[09/15/2020-17:13:02] [I] Skip inference: Disabled
[09/15/2020-17:13:02] [I] Input inference shapes: model
[09/15/2020-17:13:02] [I] === Reporting Options ===
[09/15/2020-17:13:02] [I] Verbose: Disabled
[09/15/2020-17:13:02] [I] Averages: 10 inferences
[09/15/2020-17:13:02] [I] Percentile: 99
[09/15/2020-17:13:02] [I] Dump output: Disabled
[09/15/2020-17:13:02] [I] Profile: Disabled
[09/15/2020-17:13:02] [I] Export timing to JSON file:
[09/15/2020-17:13:02] [I] Export profile to JSON file:
[09/15/2020-17:13:02] [I]
[09/15/2020-17:13:02] [E] [TRT] ../rtSafe/coreReadArchive.cpp (31) - Serialization Error in verifyHeader: 0 (Magic tag does not match)
[09/15/2020-17:13:02] [E] [TRT] INVALID_STATE: std::exception
[09/15/2020-17:13:02] [E] [TRT] INVALID_CONFIG: Deserialize the cuda engine failed.
[09/15/2020-17:13:02] [E] Engine could not be created
&&&& FAILED TensorRT.trtexec # ./trtexec --loadEngine=/home/user/disc_brand/yolov3_INT8_1_segment0.trtengine --int8
This model with FT32 precision loaded in TRT with out problem
How I can save my model to trt engine with int8 precision?