low fps on Mac m1 pro

[Aaronthecowboy]

Live video seems to be below 10fps on my Mac m1 pro. I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

[RoversX]

Live video seems to be below 10fps on my Mac m1 pro.

I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

[Aaronthecowboy]

Live video seems to be below 10fps on my Mac m1 pro. I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

Maybe letting the gpu work will improve the frame rate😂

[1057437122]

Live video seems to be below 10fps on my Mac m1 pro. I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

Maybe letting the gpu work will improve the frame rate😂

Thanks you for opensource this, but how to make GPU work? I'm using m3 pro, I just tried this, it's also very slow, like maybe 10 fps, I tried --keep-fps parameter like this python run.py --keep-fps --execution-provider coreml but it does no help 😂

[RoversX]

Live video seems to be below 10fps on my Mac m1 pro.

I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

Maybe letting the gpu work will improve the frame rate😂

I will try to quantize the model, is there any way to change the video to 540p to increase the fps

[RoversX]

I have an nvidia gpu pc, if I use the Mac camera to capture the video and do the computation in the nvidia pc, is it possible?

[Aaronthecowboy]

Live video seems to be below 10fps on my Mac m1 pro. I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

Maybe letting the gpu work will improve the frame rate😂

Thanks you for opensource this, but how to make GPU work? I'm using m3 pro, I just tried this, it's also very slow, like maybe 10 fps, I tried --keep-fps parameter like this python run.py --keep-fps --execution-provider coreml but it does no help 😂

In fact, I don’t know😂. Maybe GFPGANv1.4 and inswapper_128_fp16 need to support coreML.

[Aaronthecowboy]

Live video seems to be below 10fps on my Mac m1 pro.

I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

Maybe letting the gpu work will improve the frame rate😂

I will try to quantize the model, is there any way to change the video to 540p to increase the fps

This could totally work. You got this, bro!

[1057437122]

Live video seems to be below 10fps on my Mac m1 pro. I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

Maybe letting the gpu work will improve the frame rate😂

Thanks you for opensource this, but how to make GPU work? I'm using m3 pro, I just tried this, it's also very slow, like maybe 10 fps, I tried --keep-fps parameter like this python run.py --keep-fps --execution-provider coreml but it does no help 😂

In fact, I don’t know😂. Maybe GFPGANv1.4 and inswapper_128_fp16 need to support coreML.

sorry, I saw the author tag on your reply, I thought you are the author of this project 😂.

[Aaronthecowboy]

Live video seems to be below 10fps on my Mac m1 pro. I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

Maybe letting the gpu work will improve the frame rate😂

Thanks you for opensource this, but how to make GPU work? I'm using m3 pro, I just tried this, it's also very slow, like maybe 10 fps, I tried --keep-fps parameter like this python run.py --keep-fps --execution-provider coreml but it does no help 😂

In fact, I don’t know😂. Maybe GFPGANv1.4 and inswapper_128_fp16 need to support coreML.

sorry, I saw the author tag on your reply, I thought you are the author of this project 😂.

No biggie, our chat might just grab the author's attention😊

[1057437122]

Yeah , let’s make this issue hot so the author will give a glance to us haha

El El sáb, 10 ago 2024 a las 11:18, Aaronthecowboy @.***> escribió:

Live video seems to be below 10fps on my Mac m1 pro. I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

Maybe letting the gpu work will improve the frame rate😂

Thanks you for opensource this, but how to make GPU work? I'm using m3 pro, I just tried this, it's also very slow, like maybe 10 fps, I tried --keep-fps parameter like this python run.py --keep-fps --execution-provider coreml but it does no help 😂

In fact, I don’t know😂. Maybe GFPGANv1.4 and inswapper_128_fp16 need to support coreML.

sorry, I saw the author tag on your reply, I thought you are the author of this project 😂.

No biggie, our chat might just grab the author's attention😊

— Reply to this email directly, view it on GitHub https://github.com/hacksider/Deep-Live-Cam/issues/120#issuecomment-2280539583, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSS6ACGXBFO3NB4TA4BX33ZQXLFVAVCNFSM6AAAAABMJTRZO6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBQGUZTSNJYGM . You are receiving this because you commented.Message ID: @.***>

[RoversX]

Live video seems to be below 10fps on my Mac m1 pro.

I installed onnxruntime-silicon according to the instructions and ran the program through python run.py --execution-provider coreml, but the GPU did not work.

less than 1 fps here🤣. We might need smaller model.

Maybe letting the gpu work will improve the frame rate😂

I will try to quantize the model, is there any way to change the video to 540p to increase the fps

I changed the resolution to 360p, but it's still slow

[varna9000]

I second the terrible performance on Mac silicon. I tried reducing the resolution of OpenCV capture and fps but it doesn't influence the bad performance. I guess it's model related. If you want to experiment, check lines 279-282 of models/ui.py

[FNsi]

I don't think resolution is the key, your performance majorly depends on the model size.

[varna9000]

I've found an issue with similar symptoms. If anyone had uses onnxruntime python API before, perhaps could tell us how to add options and set COREML_FLAG_USE_CPU_ONLY = 0?

[varna9000]

The performance is better (although still choppy) with the non fp16 model

[RoversX]

The performance is better (although still choppy) with the non fp16 model

Looks great, What's the fps you got now?

[varna9000]

@RoversX I don't know how to measure them :)

[essssence]

Apple M2 - 1 FPS :( on coreml

[essssence]

@varna9000 did you change COREML_FLAG_USE_CPU_ONLY = 0 this? :)

[varna9000]

@essssence no, I downloaded the other model which is not fp16 from the same repo. Once downloaded in models/ directory, just rename it to inswapper_128_fp16.onnx

[varna9000]

Alternatively, just edit this line

[essssence]

@varna9000 tried this on GTX 1060 6GB - a bit better, but still ± around 3 FPS :D

[RoversX]

The performance is better (although still choppy) with the non fp16 model

You are right! A bit faster

[lhuanyu]

The performance is better (although still choppy) with the non fp16 model

I have tried the non fp16 model on M3 Pro Max, it's much better than the orginal model.

[ice-cap0]

non fp16 model made it go from 1fps to 15fps

[Aaronthecowboy]

The performance is better (although still choppy) with the non fp16 model

This method really worked—I got a noticeable boost in FPS (though it’s still around 10, kinda feels like the laggy days from 10 years ago).

[gongzhang]

The performance is better (although still choppy) with the non fp16 model

This method really worked—I got a noticeable boost in FPS (though it’s still around 10, kinda feels like the laggy days from 10 years ago).

It works on my M3 Max and thank you all :)

https://github.com/gongzhang/Deep-Live-Cam/blob/main/README.macos.md

[solstice-gao]

非 fp16 模型的性能更好（尽管仍然不稳定）

There is indeed an improvement in the frame rate, but it still does not reach a smooth speed. For this model, why is version 32 faster than version 16? It may be the computing architecture of apple m1, which will call more system resources at 32 precision. To solve this problem, it also needs to be transferred to gpu to run.

[109km]

I got the same problem, when not using with Face Enhancer the FPS is about 15, but when using with Face Enhancer the FPS is about 1. Does anyone know how to solve this?

[jiafanwu]

Convert the onnx model back to pytorch model and run in mps mode a bit faster in my mac M3 pro. It would actually use the GPU. One inference for the swapping cut from around 600ms to 150ms in my case. A bit better but still not smooth. For the onnxruntime, I don't think it support coreml GPU yet in python.