Benchmarking GPU vs. CPU: Unexpected Results

[NickSwardh]

Originally posted by @niclasastrom in https://github.com/NickSwardh/YoloDotNet/issues/9#issuecomment-2024052658

Performance using the GPU is worse than using the CPU. I have an RTX 4070, running Windows 11 Pro. The latest OS- and NVidia driver updates are installed.

I expected higher throughput when using the GPU, but I could be wrong. What performance can I expect, CPU vs GPU?

For example, the classification test took 130ms on the CPU and 572ms on the GPU. Do you know if this is expected?

I added a couple of lines to measure compute time:

var stopWatch=new Stopwatch();
stopWatch.Start();
List<Classification> results = yolo.RunClassification(image, 3); // Get top 5 classifications. Default = 1
stopWatch.Stop();
Console.WriteLine("Elapsed time: "+stopWatch.ElapsedMilliseconds);

Thanks for your input. If this follow-up question doesn't fit the topic, please forgive me and I will try to file my question somewhere else.