Depth value in script

[bruno-darochac]

Hi, again :x,

I would like to know, if there's an easy way to modify your script in order to replace the z value generated by the model, with the z value perceived by stereo capture. At the moment, I've implemented a queue with the stereo output in, but the number of request on the pipeline for the depth slowed the retrieval of the body data.

[geaxgx]

Hi, instead of doing N requests where each request retrieves data for 1 landmark, you can do only 1 request for the N landmarks. Currently in my code, you have: https://github.com/geaxgx/depthai_blazepose/blob/a3ce15a25c82f4663e4d263c99d7f83ece59ab64/BlazeposeDepthai.py#L430 You could have instead something like: cfg.setROIs([conf_data_1, conf_data_2, ...conf_data_N]) so with one conf_data_i for every landmark you want. It should be faster as there would be only one round-trip.

[bruno-darochac]

I'm currently using the "On Edge" version of your module, and this code is non present do I have to modify int template_manager_script.py those lines :

conf_data.roi = Rect(rect_center, rect_size)
cfg = SpatialLocationCalculatorConfig()
cfg.addROI(conf_data)

to

conf_data_1.roi = Rect(body.landmarks_1, body.landmarks_1)
conf_data_2.roi = Rect(body.landmarks_2, body.landmarks_2)
...
cfg = SpatialLocationCalculatorConfig()
cfg.addROI([conf_data_1, conf_data_2, ...])

[bruno-darochac]

I've try something else. I've linked the stereo depth capture to the script as : stereo.depth.link(manager_script.inputs['depth_data'])

And then try to get it in the script as :

${_TRACE} ("Receiving depth DATA")
depth_link = node.io['depth_data'].get()
tmp = depth_link.getFrame()

But the getFrame attribute doesn't exist for "lpb.ImgFrame", but when i look at the documentation of luxonis, I can see that in the object ImgFrame implement the method getFrame, but when I try any other method of ImgFrame it works but I'm not able to print anything, I got this error :

[18443010C1BD6F0E00] [3.1] [51.040] [Script(2)] [critical] TypeError: warn(): incompatible function arguments. The following argument types are supported:
    1. (self: lpb.Node, arg0: str) -> None

Invoked with: <lpb.Node object at 0x81023620>, 1248

Am doing it wrongly ?

[geaxgx]

You can either use: cfg.setROIs([conf_data_1, conf_data_2, ...]) or:

cfg.addROI(conf_data_1)
cfg.addROI(conf_data_2)
...

The arguments in your code: conf_data_1.roi = Rect(body.landmarks_1, body.landmarks_1) are not correct. It should be somthing like: conf_data_1.roi = Rect(rect_center_1, rect_size_1)

Why do you want to get the depth frame in the script ?

[bruno-darochac]

I want to make it in the script, because, I want the real stereo capture estimation of the landmarks in the scene. And when I've try to implement it on the host size, by just getting the depth frame and match the x y coords of the landmarks in the depth capture. I do it to frequently and so the program slowed and it was not performant. What I wanted to do by making it directly in the script, is too get the positions x, y of the var lms that you created and getting directly their depth position, instead of making tones of request on the camera. But know that I have tested more your programm, your lms generated are post-traited, and in the script they don't have the x,y frame position.

In my project, I have to generate the landmarks of the body, and by a click event, add a point on the display. The goal is to keep the point always at the same place where he was fixed on the human body. For exemple, if you put it on the arm, I defines vector between shoulder -> point and elbow -> point to keep the generate point always at the same position. Then I need the depth value, because the body can rotate and I have to respect the positon.

It was interessant for the case of my subject goes way of the camera, the size of his body change, and then my position define for a added point, depend on the first capture. Then if the size of the body change, the point will not be at the same place. And I suppose that the depth value could maybe help this kind of problem

But I think that i'll try to make it only with 2D landmarks position that seems quite good for my goal to keep it on the same place.