Which API call one has to use? to input a video from a file to further do object detection

Open snsmssss opened this issue 2 years ago • 10 comments

In the my-detection.py example part of object detection/jetson inference tutorial, it is shown how to use camera feed as follows: camera = jetson.utils.gstCamera(1280,720,"/dev/video0")

Here, instead of camera source, I want to use a video from a file say (Car.mp4) Appreciate guidance on Which API call one has to use? I tried jetson.utils.videoSource() Some times I got openGL error, some times segmentation fault.

My aim is to take a frame with a particular object of interest , out of the video file and investigate further ...

cat my-detection.py

-------------------------------------------------source code import jetson.inference import jetson.utils import argparse import sys

parse the command line

parser = argparse.ArgumentParser(description="Locate objects in a live camera stream using an object detection DNN.")

parser.add_argument("input_URI", type=str, default="", nargs='?', help="URI of the input stream") parser.add_argument("output_URI", type=str, default="", nargs='?', help="URI of the output stream") parser.add_argument("--network", type=str, default="ssd-mobilenet-v2", help="pre-trained model to load (see below for options)")

try: opt = parser.parse_known_args()[0] except: print("") parser.print_help() sys.exit(0) net = jetson.inference.detectNet("ssd-mobilenet-v2",threshold=0.5)

display =jetson.utils.glDisplay() input = jetson.utils.videoSource(opt.input_URI, argv=sys.argv)

while display.IsOpen(): img, width, height = input.Capture() detection = net.Detect(img, width, height) display.RenderOnce(img, width,height) display.SetTitle("Object Detection in the house | Network{:.0f} FPS".format(net.GetNetworkFPS()))

----------------------------------------------------------command line python3 my-detection.py CarParking.mp4 ----------------------------------------------------------output jetson.inference -- detectNet loading build-in network 'ssd-mobilenet-v2'

detectNet -- loading detection network model from: -- model networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff -- input_blob 'Input' -- output_blob 'NMS' -- output_count 'NMS_1' -- class_labels networks/SSD-Mobilenet-v2/ssd_coco_labels.txt -- threshold 0.500000 -- batch_size 1

[TRT] TensorRT version 8.0.1 [TRT] loading NVIDIA plugins... [TRT] Registered plugin creator - ::GridAnchor_TRT version 1 [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1 ...... .... ... Traceback (most recent call last): File "my-detection.py", line 28, in img, width, height = input.Capture() TypeError: 'jetson.utils.cudaImage' object is not iterable nvbuf_utils: dmabuf_fd 1277 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1279 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1281 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1283 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1285 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1287 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... [gstreamer] gstreamer mysink taglist, image-orientation=(string)rotate-90, video-codec=(string)"H.264\ (Baseline\ Profile)", bitrate=(uint)1467141, minimum-bitrate=(uint)1019940, maximum-bitrate=(uint)7481718; nvbuf_utils: dmabuf_fd 1289 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1291 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... [gstreamer] gstreamer mysink taglist, image-orientation=(string)rotate-90, video-codec=(string)"H.264\ (Baseline\ Profile)", bitrate=(uint)1467141, minimum-bitrate=(uint)878721, maximum-bitrate=(uint)7481718; nvbuf_utils: dmabuf_fd 1293 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1295 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1297 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1299 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... [gstreamer] gstDecoder -- stopping pipeline, transitioning to GST_STATE_NULL [gstreamer] gstDecoder -- onPreroll() [gstreamer] gstreamer changed state from PLAYING to PAUSED ==> capsfilter1 [gstreamer] gstreamer changed state from PLAYING to PAUSED ==> omxh264dec-omxh264dec0 [gstreamer] gstreamer changed state from PLAYING to PAUSED ==> h264parse1 [gstreamer] gstreamer changed state from PLAYING to PAUSED ==> queue0 [gstreamer] gstreamer changed state from PLAYING to PAUSED ==> qtdemux1 [gstreamer] gstreamer changed state from PLAYING to PAUSED ==> filesrc0 [gstreamer] gstreamer changed state from PLAYING to PAUSED ==> pipeline0 [gstreamer] gstDecoder -- pipeline stopped

Mar 01 '22 11:03 snsmssss

Hi @snsmssss, please see the updated tutorial here that is using the videoSource/videoOutput APIs:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-example-2.md https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/my-detection.py

Or you can just use detectnet.py and run it from video file like here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md#processing-video-files

There is additional documentation on the streaming and video API's here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md

Mar 01 '22 19:03 dusty-nv

Hello Dusty

Thank you ! Tried a few of your inputs. Will go through the latest tutorial. Will update

On Wed, Mar 2, 2022 at 1:02 AM Dustin Franklin @.***> wrote:

Hi @snsmssss https://github.com/snsmssss, please see the updated tutorial here that is using the videoSource/videoOutput APIs:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-example-2.md

https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/my-detection.py

Or you can just use detectnet.py and run it from video file like here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md#processing-video-files

There is additional documentation on the streaming and video API's here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md

— Reply to this email directly, view it on GitHub https://github.com/dusty-nv/jetson-inference/issues/1377#issuecomment-1055785778, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBKZZBOSS4A3FGLTDDSKFLU5ZWDZANCNFSM5PTYMLRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

-- Sambath Narayanan Parthasarathy Ph.D Mobile : +919500023722, +917676497952 @.*** @.*** www.linkedin.com/in/sambath-parthasarathy/ www.dataeverconsulting.com Click Here to Download the Design Thinking Workshop Brochure https://bit.ly/3DbGveC

Mar 02 '22 03:03 snsmssss

Hello Dusty After many trials, my mp4 video is playing , but object detection is wrong For some reason my video of a car and person near the car gets rotated anti clock wise 90 deg That may be the reason why the objects are not detected? I use detectnet.py without change I use mp4 video shot from my iphone

I got many alerts as given below

nvbuf_utils: dmabuf_fd 1299 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1301 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting... nvbuf_utils: dmabuf_fd 1303 mapped entry NOT found nvbuf_utils: NvReleaseFd Failed... Exiting...

I also got memory shortage warning

On Wed, Mar 2, 2022 at 1:02 AM Dustin Franklin @.***> wrote:

Hi @snsmssss https://github.com/snsmssss, please see the updated tutorial here that is using the videoSource/videoOutput APIs:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-example-2.md

https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/my-detection.py

Or you can just use detectnet.py and run it from video file like here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md#processing-video-files

There is additional documentation on the streaming and video API's here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md

— Reply to this email directly, view it on GitHub https://github.com/dusty-nv/jetson-inference/issues/1377#issuecomment-1055785778, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBKZZBOSS4A3FGLTDDSKFLU5ZWDZANCNFSM5PTYMLRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

Mar 02 '22 19:03 snsmssss

Can you try running it with --input-flip=rotate-180 ?

See here for the flags: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md#input-options

Mar 02 '22 19:03 dusty-nv

Thank you for all your pertinent inputs. I am able to get what I want. No memory shortage issues also now

I want to detect persons only ( my previous Human detection code based on TF worked on my GPU server but had severe resource constraints on Nano) so your tutorial exactly met my requirement. Now that the code is running on Nano (with the help of TRT )

One last thing ... As soon as the person is detected by the detector from a camera or .mp4 file , I want to save that frame only in the disk as, say, person.jpg containing the person with the bounding box.

To achieve this, I have been trying the following but not working

... for detection in detections: if detection.ClassID==1: output.Render(img) break ...

Any suggestions ? Is it possible to disable other objects detection with SSDMobilenet? Even simple human detection running optimally on nano will help

On Thu, Mar 3, 2022 at 1:14 AM Dustin Franklin @.***> wrote:

Can you try running it with --input-flip=rotate-180 ?

See here for the flags: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md#input-options

— Reply to this email directly, view it on GitHub https://github.com/dusty-nv/jetson-inference/issues/1377#issuecomment-1057310619, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBKZZE4KXURHWYMOOPJZS3U57AKHANCNFSM5PTYMLRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

Mar 03 '22 13:03 snsmssss

As soon as the person is detected by the detector from a camera or .mp4 file , I want to save that frame only in the disk as, say, person.jpg containing the person with the bounding box.

Hi @snsmssss, see this example script which saves a snapshot of the detected objects: https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/detectnet-snap.py

Mar 03 '22 17:03 dusty-nv

Thank you ! detectnet-snap is useful.

my next step is to upload the file to cloud

On NANO even after installing boto3,

the import boto3 fails

I am not using the container. I am running detectnet on the non container version

On Thu, Mar 3, 2022 at 11:18 PM Dustin Franklin @.***> wrote:

As soon as the person is detected by the detector from a camera or .mp4 file , I want to save that frame only in the disk as, say, person.jpg containing the person with the bounding box.

Hi @snsmssss https://github.com/snsmssss, see this example script which saves a snapshot of the detected objects: https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/detectnet-snap.py

— Reply to this email directly, view it on GitHub https://github.com/dusty-nv/jetson-inference/issues/1377#issuecomment-1058318499, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBKZZHM6GK2JL6YSBHBOULU6D3QLANCNFSM5PTYMLRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

Mar 04 '22 19:03 snsmssss

Hello Dustin

I used the python3 version. Now it is resolved Pls ignore the boto3 question

On Sat, Mar 5, 2022 at 12:55 AM Sambath Narayanan Parthasarathy < @.***> wrote:

Thank you ! detectnet-snap is useful.

my next step is to upload the file to cloud

On NANO even after installing boto3,

the import boto3 fails

I am not using the container. I am running detectnet on the non container version

On Thu, Mar 3, 2022 at 11:18 PM Dustin Franklin @.***> wrote:

As soon as the person is detected by the detector from a camera or .mp4 file , I want to save that frame only in the disk as, say, person.jpg containing the person with the bounding box.

Hi @snsmssss https://github.com/snsmssss, see this example script which saves a snapshot of the detected objects: https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/detectnet-snap.py

— Reply to this email directly, view it on GitHub https://github.com/dusty-nv/jetson-inference/issues/1377#issuecomment-1058318499, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBKZZHM6GK2JL6YSBHBOULU6D3QLANCNFSM5PTYMLRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

-- Sambath Narayanan Parthasarathy Ph.D Mobile : +919500023722, +917676497952 @.*** @.*** www.linkedin.com/in/sambath-parthasarathy/ www.dataeverconsulting.com Click Here to Download the Design Thinking Workshop Brochure https://bit.ly/3DbGveC

Mar 04 '22 19:03 snsmssss

Hello Dusty

I have put together a Burglary Prevention application using /customizing detectnet-snap.py code

I want to submit this as an solution / blog for AI-Jetson-nano certification

I will be acknowledging NVIDIA and you for all the support

Can I share the modified code in github - with proper ack.

Can I do ? Do I need to take care of anything else ?

Thank you for your guidance

On Wed, Mar 2, 2022 at 1:02 AM Dustin Franklin @.***> wrote:

Hi @snsmssss https://github.com/snsmssss, please see the updated tutorial here that is using the videoSource/videoOutput APIs:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-example-2.md

https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/my-detection.py

Or you can just use detectnet.py and run it from video file like here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md#processing-video-files

There is additional documentation on the streaming and video API's here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md

— Reply to this email directly, view it on GitHub https://github.com/dusty-nv/jetson-inference/issues/1377#issuecomment-1055785778, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBKZZBOSS4A3FGLTDDSKFLU5ZWDZANCNFSM5PTYMLRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

Mar 10 '22 12:03 snsmssss

Sure no problem, you can put it on GitHub

From: Sambath Parthasarathy @.> Sent: Thursday, March 10, 2022 7:12 AM To: dusty-nv/jetson-inference @.> Cc: Dustin Franklin @.>; Comment @.> Subject: Re: [dusty-nv/jetson-inference] Which API call one has to use? to input a video from a file to further do object detection (Issue #1377)

Hello Dusty

I have put together a Burglary Prevention application using /customizing detectnet-snap.py code

I want to submit this as an solution / blog for AI-Jetson-nano certification

I will be acknowledging NVIDIA and you for all the support

Can I share the modified code in github - with proper ack.

Can I do ? Do I need to take care of anything else ?

Thank you for your guidance

On Wed, Mar 2, 2022 at 1:02 AM Dustin Franklin @.***mailto:***@***.***> wrote:

Hi @snsmssss https://github.com/snsmssss, please see the updated tutorial here that is using the videoSource/videoOutput APIs:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-example-2.md

https://github.com/dusty-nv/jetson-inference/blob/master/python/examples/my-detection.py

Or you can just use detectnet.py and run it from video file like here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md#processing-video-files

There is additional documentation on the streaming and video API's here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md

Reply to this email directly, view it on GitHub https://github.com/dusty-nv/jetson-inference/issues/1377#issuecomment-1055785778, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBKZZBOSS4A3FGLTDDSKFLU5ZWDZANCNFSM5PTYMLRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***mailto:***@***.***>

-- Sambath Narayanan Parthasarathy Ph.D Mobile : +919500023722, +917676497952 @.mailto:***@***.*** @.mailto:***@***.*** www.linkedin.com/in/sambath-parthasarathy/http://www.linkedin.com/in/sambath-parthasarathy/ www.dataeverconsulting.comhttp://www.dataeverconsulting.com Click Here to Download the Design Thinking Workshop Brochure https://bit.ly/3DbGveC

Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdusty-nv%2Fjetson-inference%2Fissues%2F1377%23issuecomment-1063989529&data=04%7C01%7Cdustinf%40nvidia.com%7C442fb3d6b1634731896108da028f3215%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637825111301687969%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kURuXNLJZSzwy2IlyJ%2F%2BftVmizjisOJzdbia6yqTels%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADVEGK4MSYZH7HPBKCO3EZDU7HRJNANCNFSM5PTYMLRQ&data=04%7C01%7Cdustinf%40nvidia.com%7C442fb3d6b1634731896108da028f3215%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637825111301687969%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=B6TflThdP8XFvouLqa35F901fkqKUrELWUSDk6d154A%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cdustinf%40nvidia.com%7C442fb3d6b1634731896108da028f3215%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637825111301687969%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pB0UN9adfGf%2FfKuThc3QQ23J5oMqrI2c2tyQaEKJnlE%3D&reserved=0 or Androidhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cdustinf%40nvidia.com%7C442fb3d6b1634731896108da028f3215%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637825111301687969%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=exwDkpu%2Bz9SfJP7ZVsCSyNiGLo7PTXKs43y%2FmsqGQMw%3D&reserved=0. You are receiving this because you commented.Message ID: @.@.>>

Mar 10 '22 16:03 dusty-nv

jetson-inference jetson-inference copied to clipboard

Which API call one has to use? to input a video from a file to further do object detection

My aim is to take a frame with a particular object of interest , out of the video file and investigate further ...

parse the command line

jetson-inference
jetson-inference copied to clipboard