Performance problem

[Alex18947]

Hi,

thanks for the nice work, it seems however, that there is still a performance problem when running on GPU, i.e. the code cannot fully utilize the GPU and has high CPU usage when running on GPU. Steps to reproduce:

• modify the code so that you create an endless loop around these two calls - net.forward and net.write_results,

• on my GPU the GPU load goes up to 60 - 70 percent, not higher and CPU goes up to 20 percent.

Expected result: GPU load should go up to 100 percent and CPU usage should be small.

Any thoughts on the CPU consumption? It also might be a libtorch issue though...

Thanks.

[walktree]

Hi Alex, I have not tested yet, but high CPU usage might be caused by function 'write_results', since a few lines of code in this method is running on CPU. You might notice line 763.

You could try to modify these lines of code and test again.

[Alex18947]

Hi walktree,

thanks for your comment. No, that's not the case. Even if you create a loop like this in your code:

while (1) { output = net.forward(img_var); }

the GPU usage goes only to 60 percent and CPU is still high. Testing with original Darknet Yolov3 - you can easily use 100 percent of the GPU. So not being able to fully utilize the GPU is a bottleneck of this libtorch fork.

I am currently testing on Windows with latest libtorch build.

Alex

[Alex18947]

Ok, looking into this a bit more, it seems that the CPU is being consumed by the Darknet::forward implementation. In a standard case you would chain SequentialImpl and call forward on the entire model once. Here, however, you are calling forward for each module and, therefore, you are synchronizing to CPU after each such call. This increases the CPU usage and prevents the GPU to be fully utilized.

A.

[Alex18947]

Just FYI, I believe to know what's happening here. See the discussion here.

A.

[walktree]

@Alex18947 , thanks for your information. I will do some testing on the CPU consumption later and let you know if I get something new.