libtorch性能问题调研

[gemfield]

测试步骤和方法

使用https://zhuanlan.zhihu.com/p/363319763 中的：

• "PyTorch resnet50 benchmark步骤"
• "LibTorch resnet50 benchmark步骤"

注意：MLab HomePod 1.0中的PyTorch版本为1.8.1

测试模型：resnet50

环境

宿主机OS：Ubuntu 20.04 软件环境：MLab HomePod 1.0 CPU：Intel(R) Core(TM) i9-9820X CPU @ 3.30GHz GPU：NVIDIA GTX 2080ti GPU驱动：NVIDIA-SMI 450.102.04 Driver Version: 450.102.04 CUDA Version: 11.0

测试场景	CPU利用率	内存(GB)	GPU利用率	显存(GB)	线程数
C++ libtorch	~105%	~4.1	100%	~3.7	25
PyTorch	~132%	~3.9	~95%	~9	32

input shape = 224x224

测试场景	forward time(ms)
C++ libtorch	3.950
PyTorch	6.4478

input shape = 640x640

测试场景	forward time(ms)
C++ libtorch	11.242
PyTorch	10.4656

input shape = 1280x720

测试场景	forward time(ms)
C++ libtorch	24.678
PyTorch	20.8056

input shape = 1280x1280

测试场景	forward time(ms)
C++ libtorch	38.094
PyTorch	37.6037

[gemfield]

测试模型：retinaface(backbone: resnet50)

环境

宿主机OS：Ubuntu 20.04 软件环境：MLab HomePod 1.0 CPU：Intel(R) Core(TM) i9-9820X CPU @ 3.30GHz GPU：NVIDIA GTX 2080ti GPU驱动：NVIDIA-SMI 450.102.04 Driver Version: 450.102.04 CUDA Version: 11.0

测试场景	CPU利用率	内存(GB)	GPU利用率	显存(GB)	线程数
C++ libtorch	~105%	~4.1	100%	~3.8	25
PyTorch	~170%	~4.1	~99%	~10	32

input shape = 224x224

测试场景	forward time(ms)
C++ libtorch	6.640
PyTorch	11.3491

input shape = 640x640

测试场景	forward time(ms)
C++ libtorch	16.899
PyTorch	13.5242

input shape = 1280x720

测试场景	forward time(ms)
C++ libtorch	33.532
PyTorch	24.9352

input shape = 1280x1280

测试场景	forward time(ms)
C++ libtorch	50.001
PyTorch	44.2599

[gemfield]

测试模型：ResNet50 CPU

环境

宿主机OS：Ubuntu 20.04 软件环境：MLab HomePod 1.0 CPU：Intel(R) Core(TM) i9-9820X CPU @ 3.30GHz RAM: 64G

测试场景	CPU利用率	内存(GB)	线程数
C++ libtorch	~1001%	~5.5	32
PyTorch	~978%	~11.3	32

input shape = 224x224

测试场景	forward time(ms)
C++ libtorch	26.988
PyTorch	40.2075

input shape = 640x640

测试场景	forward time(ms)
C++ libtorch	167.991
PyTorch	231.1375

input shape = 1280x720

测试场景	forward time(ms)
C++ libtorch	419.165
PyTorch	520.6793

input shape = 1280x1280

测试场景	forward time(ms)
C++ libtorch	862.972
PyTorch	1014.6398

[liweizheng0528]

请问你的libtorch和cuda版本是多少，我在ubuntu20.04+libtorch1.9.0+cuda11.1上torch::cuda::is_available()返回false，是什么原因呢？

[atztao]

这个库的大小大该占多少空间