vgg-benchmarks
vgg-benchmarks copied to clipboard
aizvorski
More information
Great initiative!
As I get it from your code these are all training times, right? If that is the case could it be mentioned somewhere? I think also adding percentages (fastest: 100%, then other lower %s)
Also, what about adding "application times"? ie: for each framework, benchmark the time it takes to apply the trained data on some input data (probably from the test set).
Also, time is nice but what about memory/IOps/CPU GPU%/...
@fenollp Thank you!
Yes these are all training times (forward + backward + weights update). Some other benchmarks only quote the forward+backward pass. I will change the description to make that clearer.
It would be nice to have "application times" but I wasn't focusing on that initially.
Memory, cpu, GPU% are all interesting. I have GPU memory % and GPU compute % results in the results/*dmon files, but I haven't made them into a table yet. All of these frameworks in practice seem to maximize use of the GPU compute, but they use very different amounts of memory.
