jetson-inference
jetson-inference copied to clipboard
Published
20 hours ago •
dusty-nv
dusty-nv
Use the detectnet command to detect the video, the result of the detect video is distorted
Command: detectnet ./combine.avi
The output:
Source video:
Complete output:
(rgzy5) root@rgzy:/rg/8fusion-cpp/gst-cv-video-src/build# detectnet ./combine.avi
[gstreamer] initialized gstreamer, version 1.16.3.0
[gstreamer] gstDecoder -- creating decoder for ./combine.avi
[gstreamer] gstDecoder -- discovered video resolution: 1818x452 (framerate 25.000000 Hz)
[gstreamer] gstDecoder -- discovered video caps: image/jpeg, framerate=(fraction)25/1, width=(int)1818, height=(int)452
[gstreamer] gstDecoder -- pipeline string:
[gstreamer] filesrc location=./combine.avi ! avidemux ! queue ! jpegdec name=decoder ! video/x-raw ! appsink name=mysink
[video] created gstDecoder from file:///rg/8fusion-cpp/gst-cv-video-src/build/./combine.avi
------------------------------------------------
gstDecoder video options:
------------------------------------------------
-- URI: file:///rg/8fusion-cpp/gst-cv-video-src/build/./combine.avi
- protocol: file
- location: ./combine.avi
- extension: avi
-- deviceType: file
-- ioType: input
-- codec: MJPEG
-- codecType: cpu
-- width: 1818
-- height: 452
-- frameRate: 25
-- numBuffers: 4
-- zeroCopy: true
-- flipMethod: none
-- loop: 0
------------------------------------------------
[OpenGL] glDisplay -- X screen 0 resolution: 2560x1440
[OpenGL] glDisplay -- X window resolution: 2560x1440
[OpenGL] glDisplay -- display device initialized (2560x1440)
[video] created glDisplay from display://0
------------------------------------------------
glDisplay video options:
------------------------------------------------
-- URI: display://0
- protocol: display
- location: 0
-- deviceType: display
-- ioType: output
-- width: 2560
-- height: 1440
-- frameRate: 0
-- numBuffers: 4
-- zeroCopy: true
------------------------------------------------
detectNet -- loading detection network model from:
-- model networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
-- input_blob 'Input'
-- output_blob 'NMS'
-- output_count 'NMS_1'
-- class_labels networks/SSD-Mobilenet-v2/ssd_coco_labels.txt
-- threshold 0.500000
-- batch_size 1
[TRT] TensorRT version 8.5.2
[TRT] loading NVIDIA plugins...
[TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[TRT] Registered plugin creator - ::BatchedNMS_TRT version 1
[TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1
[TRT] Registered plugin creator - ::Clip_TRT version 1
[TRT] Registered plugin creator - ::CoordConvAC version 1
[TRT] Registered plugin creator - ::CropAndResizeDynamic version 1
[TRT] Registered plugin creator - ::CropAndResize version 1
[TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1
[TRT] Registered plugin creator - ::DetectionLayer_TRT version 1
[TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[TRT] Registered plugin creator - ::EfficientNMS_TRT version 1
[TRT] Could not register plugin creator - ::FlattenConcat_TRT version 1
[TRT] Registered plugin creator - ::GenerateDetection_TRT version 1
[TRT] Registered plugin creator - ::GridAnchor_TRT version 1
[TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1
[TRT] Registered plugin creator - ::GroupNorm version 1
[TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1
[TRT] Registered plugin creator - ::InstanceNormalization_TRT version 2
[TRT] Registered plugin creator - ::LayerNorm version 1
[TRT] Registered plugin creator - ::LReLU_TRT version 1
[TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[TRT] Registered plugin creator - ::NMSDynamic_TRT version 1
[TRT] Registered plugin creator - ::NMS_TRT version 1
[TRT] Registered plugin creator - ::Normalize_TRT version 1
[TRT] Registered plugin creator - ::PillarScatterPlugin version 1
[TRT] Registered plugin creator - ::PriorBox_TRT version 1
[TRT] Registered plugin creator - ::ProposalDynamic version 1
[TRT] Registered plugin creator - ::ProposalLayer_TRT version 1
[TRT] Registered plugin creator - ::Proposal version 1
[TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1
[TRT] Registered plugin creator - ::Region_TRT version 1
[TRT] Registered plugin creator - ::Reorg_TRT version 1
[TRT] Registered plugin creator - ::ResizeNearest_TRT version 1
[TRT] Registered plugin creator - ::ROIAlign_TRT version 1
[TRT] Registered plugin creator - ::RPROI_TRT version 1
[TRT] Registered plugin creator - ::ScatterND version 1
[TRT] Registered plugin creator - ::SeqLen2Spatial version 1
[TRT] Registered plugin creator - ::SpecialSlice_TRT version 1
[TRT] Registered plugin creator - ::SplitGeLU version 1
[TRT] Registered plugin creator - ::Split version 1
[TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1
[TRT] detected model format - UFF (extension '.uff')
[TRT] desired precision specified for GPU: FASTEST
[TRT] requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT] [MemUsageChange] Init CUDA: CPU +221, GPU +0, now: CPU 258, GPU 4196 (MiB)
[TRT] Trying to load shared library libnvinfer_builder_resource.so.8.5.2
[TRT] Loaded shared library libnvinfer_builder_resource.so.8.5.2
[TRT] [MemUsageChange] Init builder kernel library: CPU +302, GPU +285, now: CPU 583, GPU 4502 (MiB)
[TRT] native precisions detected for GPU: FP32, FP16, INT8
[TRT] selecting fastest native precision for GPU: FP16
[TRT] found engine cache file /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.1.1.8502.GPU.FP16.engine
[TRT] found model checksum /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.sha256sum
[TRT] echo "$(cat /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.sha256sum) /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff" | sha256sum --check --status
[TRT] model matched checksum /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.sha256sum
[TRT] loading network plan from engine cache... /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.1.1.8502.GPU.FP16.engine
[TRT] device GPU, loaded /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
[TRT] Loaded engine size: 33 MiB
[TRT] Trying to load shared library libcublas.so.11
[TRT] Loaded shared library libcublas.so.11
[TRT] Using cublas as plugin tactic source
[TRT] Trying to load shared library libcublasLt.so.11
[TRT] Loaded shared library libcublasLt.so.11
[TRT] Using cublasLt as core library tactic source
[TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +534, GPU +379, now: CPU 871, GPU 4917 (MiB)
[TRT] Trying to load shared library libcudnn.so.8
[TRT] Loaded shared library libcudnn.so.8
[TRT] Using cuDNN as plugin tactic source
[TRT] Using cuDNN as core library tactic source
[TRT] [MemUsageChange] Init cuDNN: CPU +83, GPU +42, now: CPU 954, GPU 4959 (MiB)
[TRT] Deserialization required 1215333 microseconds.
[TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +32, now: CPU 0, GPU 32 (MiB)
[TRT] Trying to load shared library libcublas.so.11
[TRT] Loaded shared library libcublas.so.11
[TRT] Using cublas as plugin tactic source
[TRT] Trying to load shared library libcublasLt.so.11
[TRT] Loaded shared library libcublasLt.so.11
[TRT] Using cublasLt as core library tactic source
[TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 954, GPU 4959 (MiB)
[TRT] Trying to load shared library libcudnn.so.8
[TRT] Loaded shared library libcudnn.so.8
[TRT] Using cuDNN as plugin tactic source
[TRT] Using cuDNN as core library tactic source
[TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 954, GPU 4959 (MiB)
[TRT] Total per-runner device persistent memory is 9216
[TRT] Total per-runner host persistent memory is 159632
[TRT] Allocated activation device memory of size 7000064
[TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +7, now: CPU 0, GPU 39 (MiB)
[TRT]
[TRT] CUDA engine context initialized on device GPU:
[TRT] -- layers 106
[TRT] -- maxBatchSize 1
[TRT] -- deviceMemory 7000064
[TRT] -- bindings 3
[TRT] binding 0
-- index 0
-- name 'Input'
-- type FP32
-- in/out INPUT
-- # dims 3
-- dim #0 3
-- dim #1 300
-- dim #2 300
[TRT] binding 1
-- index 1
-- name 'NMS'
-- type FP32
-- in/out OUTPUT
-- # dims 3
-- dim #0 1
-- dim #1 100
-- dim #2 7
[TRT] binding 2
-- index 2
-- name 'NMS_1'
-- type FP32
-- in/out OUTPUT
-- # dims 3
-- dim #0 1
-- dim #1 1
-- dim #2 1
[TRT]
[TRT] binding to input 0 Input binding index: 0
[TRT] binding to input 0 Input dims (b=1 c=3 h=300 w=300) size=1080000
[TRT] binding to output 0 NMS binding index: 1
[TRT] binding to output 0 NMS dims (b=1 c=1 h=100 w=7) size=2800
[TRT] binding to output 1 NMS_1 binding index: 2
[TRT] binding to output 1 NMS_1 dims (b=1 c=1 h=1 w=1) size=4
[TRT]
[TRT] device GPU, /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff initialized.
[TRT] W = 7 H = 100 C = 1
[TRT] detectNet -- maximum bounding boxes: 100
[TRT] loaded 91 class labels
[TRT] detectNet -- number of object classes: 91
[TRT] loaded 0 class colors
[TRT] didn't load expected number of class colors (0 of 91)
[TRT] filling in remaining 91 class colors with default colors
[gstreamer] opening gstDecoder for streaming, transitioning pipeline to GST_STATE_PLAYING
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0
[gstreamer] gstreamer changed state from NULL to READY ==> decoder
[gstreamer] gstreamer changed state from NULL to READY ==> queue0
[gstreamer] gstreamer changed state from NULL to READY ==> avidemux1
[gstreamer] gstreamer changed state from NULL to READY ==> filesrc0
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0
[gstreamer] gstreamer changed state from READY to PAUSED ==> decoder
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> queue0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer stream status CREATE ==> sink
[gstreamer] gstreamer changed state from READY to PAUSED ==> avidemux1
[gstreamer] gstreamer changed state from READY to PAUSED ==> filesrc0
[gstreamer] gstreamer stream status ENTER ==> sink
[gstreamer] gstDecoder -- onPreroll()
[gstreamer] gstreamer message stream-start ==> pipeline0
[gstreamer] gstreamer mysink taglist, video-codec=(string)"Motion\ JPEG", bitrate=(uint)7547532;
[gstreamer] gstreamer mysink taglist, encoder=(string)Lavf58.29.100, container-format=(string)AVI;
[gstreamer] gstBufferManager -- map buffer size was less than max size (1234864 vs 1234871)
[gstreamer] gstBufferManager recieve caps: video/x-raw, format=(string)I420, width=(int)1818, height=(int)452, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)25/1
[gstreamer] gstBufferManager -- recieved first frame, codec=MJPEG format=i420 width=1818 height=452 size=1234871
[cuda] allocated 4 ring buffers (1234871 bytes each, 4939484 bytes total)
[cuda] allocated 4 ring buffers (8 bytes each, 32 bytes total)
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[gstreamer] gstreamer message async-done ==> pipeline0
[gstreamer] gstreamer message new-clock ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> decoder
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> queue0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> avidemux1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> filesrc0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0
[cuda] allocated 4 ring buffers (2465208 bytes each, 9860832 bytes total)
[OpenGL] glDisplay -- set the window size to 1818x452
[OpenGL] creating 1818x452 texture (GL_RGB8 format, 2465208 bytes)
[cuda] registered openGL texture for interop access (1818x452, GL_RGB8, 2465208 bytes)
[TRT] ------------------------------------------------
[TRT] Timing Report /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
[TRT] ------------------------------------------------
[TRT] Pre-Process CPU 0.04451ms CUDA 0.08986ms
[TRT] Network CPU 7.50483ms CUDA 7.01840ms
[TRT] Post-Process CPU 0.01373ms CUDA 0.05536ms
[TRT] Total CPU 7.56307ms CUDA 7.16362ms
[TRT] ------------------------------------------------
[TRT] note -- when processing a single image, run 'sudo jetson_clocks' before
to disable DVFS for more accurate profiling/timing measurements
1 objects detected
detected obj 0 class #5 (airplane) confidence=0.747070
bounding box 0 (12.43, 0.33) (1738.11, 451.00) w=1725.68 h=450.67
[TRT] ------------------------------------------------
[TRT] Timing Report /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
[TRT] ------------------------------------------------
[TRT] Pre-Process CPU 0.02781ms CUDA 0.09504ms
[TRT] Network CPU 11.44885ms CUDA 7.59594ms
[TRT] Post-Process CPU 0.03040ms CUDA 0.00227ms
[TRT] Visualize CPU 3.08336ms CUDA 3.08707ms
[TRT] Total CPU 14.59042ms CUDA 10.78032ms
[TRT] ------------------------------------------------
1 objects detected
detected obj 0 class #5 (airplane) confidence=0.610840
bounding box 0 (10.65, 0.66) (1725.68, 451.78) w=1715.03 h=451.12
[TRT] ------------------------------------------------
[TRT] Timing Report /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
[TRT] ------------------------------------------------
[TRT] Pre-Process CPU 0.02509ms CUDA 0.08838ms
[TRT] Network CPU 8.83468ms CUDA 6.72522ms
[TRT] Post-Process CPU 0.01286ms CUDA 0.01312ms
[TRT] Visualize CPU 0.05565ms CUDA 0.63891ms
[TRT] Total CPU 8.92828ms CUDA 7.46563ms
[TRT] ------------------------------------------------
@bladetin does video-viewer or video-viewer.py display your video properly?
If not, I think it may have to do with it being an uncommon resolution (1818x452) and MJPEG. Could you try a different video or resizing it with a different tool, so see if it has an impact?
Also, can you try video-viewer --headless combine.avi combine.mp4 and see if it transcodes the video properly? If so, the distortion is present in the OpenGL visualization and not the actual processing.
