mtcnn_facenet_cpp_tensorRT icon indicating copy to clipboard operation
mtcnn_facenet_cpp_tensorRT copied to clipboard

Deepstream

Open hirwa145 opened this issue 4 years ago • 95 comments

Is there a way i can deploy using Nvidia Deepstream ? Or create Deepstream app from this?

hirwa145 avatar Feb 01 '21 05:02 hirwa145

check this, dlib model and facenet have similar output https://forums.developer.nvidia.com/t/face-recognition-using-dlib-with-jetson-nano/167172

shubham-shahh avatar Feb 02 '21 11:02 shubham-shahh

So it is possible

hirwa145 avatar Feb 02 '21 15:02 hirwa145

@hirwa145 this was asked before and i think it is a great idea. Let me know if you are interested to work on that

nwesem avatar Feb 05 '21 11:02 nwesem

@nwesem I am working on a similar project, but first I'm trying to use dlib's model for face recognition with deepstream

shubham-shahh avatar Feb 05 '21 18:02 shubham-shahh

Have you succeeded with it?

hirwa145 avatar Feb 10 '21 06:02 hirwa145

currently i am working on https://github.com/riotu-lab/deepstream-facenet this, because to run this repo with deepstream will require us to configure mtcnn as with the deepstream.

shubham-shahh avatar Feb 10 '21 07:02 shubham-shahh

Hey, I managed to get facenet working with deepstream but it's a python implementation. shall I put up a pull request? @nwesem

shubham-shahh avatar Feb 13 '21 04:02 shubham-shahh

@shubham-shahh yes please. Can you upload it. I am very interested to see how you managed to make it work with Deepstream

hirwa145 avatar Feb 15 '21 11:02 hirwa145

I have a python edition, currently I am working on a C++ version as well, shall I PR a python version?

shubham-shahh avatar Feb 15 '21 12:02 shubham-shahh

Yes, you can PR the python version

hirwa145 avatar Feb 15 '21 12:02 hirwa145

@shubham-shahh you pull requested the python version?

hirwa145 avatar Feb 15 '21 22:02 hirwa145

not yet, @nwesem should create a new branch for the python version, meanwhile, I'll commit the project to my account.

shubham-shahh avatar Feb 16 '21 02:02 shubham-shahh

Yeah. That is also okay, you can commit it to your account whilest we are waiting for him to open new branch

hirwa145 avatar Feb 16 '21 09:02 hirwa145

Yeah. That is also okay, you can commit it to your account whilest we are waiting for him to open new branch

I'll complete it by this weekend

shubham-shahh avatar Feb 16 '21 12:02 shubham-shahh

Alright, thank you

hirwa145 avatar Feb 16 '21 13:02 hirwa145

hey @shubham-shahh, cool! Thanks for contributing to this project! I am looking forward to see your implementation. I added you as a collaborator on the project, please add your code to a branch called python-develop and I will check it out and merge it if everything works as expected. Currently I am pretty busy with work, but I will take the time to test your implementation asap. If you can't create a new branch just push it to your forked repo and send me a pull request

nwesem avatar Feb 17 '21 13:02 nwesem

hey @shubham-shahh, cool! Thanks for contributing to this project! I am looking forward to see your implementation. I added you as a collaborator on the project, please add your code to a branch called python-develop and I will check it out and merge it if everything works as expected. Currently I am pretty busy with work, but I will take the time to test your implementation asap.

Hey, I hope you are doing fine, thanks for the branch I'll submit my work by this weekend.

shubham-shahh avatar Feb 17 '21 13:02 shubham-shahh

Hey, @nwesem I updated the develop branch of my fork with the python implementation, @hirwa145 please test it and let me know if there are any issues, and once this is tested ill put up a pull request on your develop branch and it would be great if we keep them in separate branches. Here

shubham-shahh avatar Feb 20 '21 12:02 shubham-shahh

Hello @shubham-shahh . Great job doing deepstream implementation. So far i tried the python implementation and it is working perfect and thank you. I am currently testing the cpp implementation version even tho it is taking long for testing the sample app and i am not sure why. But i have a question, is there a way to implement the recognition where faces in video files are compared to saved images in the folder and display the name of the person instead of '"person". Thank you.

hirwa145 avatar Feb 23 '21 12:02 hirwa145

Hello @shubham-shahh . Great job doing deepstream implementation. So far i tried the python implementation and it is working perfect and thank you. I am currently testing the cpp implementation version even tho it is taking long for testing the sample app and i am not sure why. But i have a question, is there a way to implement the recognition where faces in video files are compared to saved images in the folder and display the name of the person instead of '"person". Thank you.

Hello, did you follow the guide step by step? was there any issue at any step? and thanks for testing. and about your question, I will implement that functionality this weekend if possible but if you want to implement it by yourself i can tell you the approach which you can follow. thanks again

shubham-shahh avatar Feb 23 '21 12:02 shubham-shahh

for python implementation, everything works as expected. But for C++ implementation, i have to first test the test sample, so it seems it stuck here NvMMLiteOpen : Block : BlockType = 261 NVMEDIA: Reading vendor.tegra.display-size : status: 6 NvMMLiteBlockCreate : Block : BlockType = 261 . D you know why?

hirwa145 avatar Feb 23 '21 12:02 hirwa145

for python implementation, everything works as expected. But for C++ implementation, i have to first test the test sample, so it seems it stuck here NvMMLiteOpen : Block : BlockType = 261 NVMEDIA: Reading vendor.tegra.display-size : status: 6 NvMMLiteBlockCreate : Block : BlockType = 261 . D you know why?

Is this the sample App? or the one i included in the repo? whats your video source .mp4 or .h264?

shubham-shahh avatar Feb 23 '21 12:02 shubham-shahh

sample_720p.mp4, as mentioned from your instructions

hirwa145 avatar Feb 23 '21 12:02 hirwa145

sample_720p.mp4, as mentioned from your instructions

and is it the sample app present already or you are using the repo app?

shubham-shahh avatar Feb 23 '21 12:02 shubham-shahh

yes, available since i installed deepstream. it must be tested on that sample? Or i can test on another .mp4 file?

hirwa145 avatar Feb 23 '21 12:02 hirwa145

yes, available since i installed deepstream. it must be tested on that sample? Or i can test on another .mp4 file?

The one that is already presemt works with .h264 files only and the one in the repo works with mp4 files and rtsp streams

shubham-shahh avatar Feb 23 '21 12:02 shubham-shahh

So which means when i am doing testing c++ implementation. i do this ./deepstream-infer-tensor-meta-app -t infer /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.h264 instead of ./deepstream-infer-tensor-meta-app -t infer /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4

hirwa145 avatar Feb 23 '21 12:02 hirwa145

So which means when i am doing testing c++ implementation. i do this ./deepstream-infer-tensor-meta-app -t infer /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.h264 instead of ./deepstream-infer-tensor-meta-app -t infer /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4

right

shubham-shahh avatar Feb 23 '21 12:02 shubham-shahh

Perfect. Now i am able to run C++ implementation successfully and detecting faces as it should. Sometimes misses face that are out of focus.( probably due to resolution/quality of the video) , also if i noticed that when testing on .mp4 or longer video, there is no quiting option, so i have to wait to wait for video to finish so that i quit the app.

hirwa145 avatar Feb 23 '21 13:02 hirwa145

Also if it is possible, could you show me how i can implement the function that the name of face owner is displayed please? Thank you

hirwa145 avatar Feb 23 '21 13:02 hirwa145