`gen2-face-recognition` is too heavy

[kazyam53]

gen2-face-recognition demo is too heavy, delayed and crash soon on my OAK-D and OAK-D Lite. I rollback to the Initial commit 1b532c0ebec4b9e45ae4fa79b5f36ada7817a780 and try then it's stable.

[kazyam53]

I confirmed that is fast and stable until 87c2f440f309671c40d62d665b5d8b1dbbaf13c9.

[Erol444]

Hello @kazyam53 , Which version of the depthai library are you using? Did you install requirements.txt? Thanks, Erik

[kazyam53]

I installed requirement.txt, have confirmed I'm using 2.15.2. Especially the time it detect 2 or more persons, it break.

[VanDavv]

I just tested the latest version on OAK-D-Lite and the same version:

• I see the delay/lag that is happening pretty often, most likely due to some computations on the host (@Erol444 could it be host side sync?). It's visible both during the learning process (with --name param) and on a regular detection run
• I didn't experience any crashes though, both learning and regular run worked pretty stably (tested for around 10 minutes and no crash, tried for over a minute with two faces visible simultaneously, and also no crash experienced. Could you paste your console output @kazyam53?

[kazyam53]

@VanDavv Thank you for reply. In my environment, it completely stacked and not updated anymore when it detect 4 or more faces. Nothing is output on console, just stopped processing new image.

[VanDavv]

Ok, that's strange. Which OS/Python version are you using?

[kazyam53]

I tried on 2 PCs, but same OS and python ver. OS version: Ubuntu 20.04 Python: 3.8.10

[Erol444]

Hello @kazyam53 , That is to be expected, by default queue size is 4 messages, so 4+ faces will make it freeze. Also the latency/FPS drop at 3 faces already will be quite bad. You could consider using different AI model (one that expects multiple faces at the same time) or, if you are OK with low FPS/higher latency, increase queue size. Thanks, Erik

[kazyam53]

Thanks, @Erol444 ! Actually I don't need to detect so many faces, just want to avoid the risk of freezing. It's better for me to be able to limit the max number of faces to detect and pass to recognition, anyway I try to change que size or FPS at first.

[Erol444]

Hello @kazyam53 , you could easily limit the amount of faces to be recognized. You can set that inside Script node, in the for loop (code line here) that loops through all face detections and continues with the recognition process on these frames. So if you want to only recognize first frame, just add brake inside the for loop linked above. Thanks, Erik

[kazyam53]

@Erol444 Sorry for late. I added break at here but it freeze as soon as it detect 2 or more faces.

[Erol444]

Hi @kazyam53 , I forgot to mention that you would need to remove other detections as well. When adding detections (here), you should limit number of detections inside ImgDetections msg that are actually saved. Otherwise the Script node will wait for other inferences and block/freeze. Thanks, Erik

[kazyam53]

Thanks @Erol444, I added removing other detections, it can avoid freeze and detect 1 face but it's still unstable (sometimes cause short stack). I'll try to modify other part or change face detection model to limit the number of detections.

[kazyam53]

I set colorCamera fps at 15, it became stable until 4 faces on OAK-D lite. But to be perfectly stable, it should set limit of input numbers to recognition. Can I limit a number of msg.detections before add_msg here? https://github.com/luxonis/depthai-experiments/blob/61ff41c5235220683e7dd4c677b1438b1d1fe785/gen2-face-recognition/main.py#L220

I tried to delete list elements but it seems impossible.

 msg = q.get()
    if(name == "detection"):
        if len(msg.detections) > 1:
            del msg.detections[1:]
    sync.add_msg(msg, name)

[Luxonis-Brandon]

Thanks. Good to know and CC: @Erol444 on this as I'm not sure.

[Erol444]

Hi @kazyam53 , you can do that with python subarray; msg.detections[:4].

[kazyam53]

Thanks @Erol444! I could limit number of faces to input to recognition, but it's still stack when it detect 6 or more faces. I suspected recognition part cause freeze, but it seems that face detection itself is heavy so I may need to change the model of face detection to make this stable.

[Erol444]

Hi @kazyam53 , what do you mean by stack? So it just slows down a lot (FPS)?

[kazyam53]

@Erol444 No, completely froze and not recovered.

[Erol444]

@kazyam53 have you limited number of frames both on device (script node) and on the host (in syncing helper)?

[kazyam53]

All I did are:

1. Limit color camera FPS at 15 by cam.setFps(15).
2. Limit the number of detections to add to msg here

            if q.has():
                msg = q.get()
                if(name == "detection"):
                    if len(msg.detections) > 4:
                        msg.detections = msg.detections[:4]
                sync.add_msg(msg, name)

3. Limit the number of faces in the messege (but it may no need.) here

    if face_dets is not None:
        if len(face_dets.detections) > 4:
            face_dets.detections[:4]
        # node.warn(f"New detection start")