pytorch
Make the C++ backend of the torchvision wheel usable for C++ development
🚀 The feature
Currently, the torchvision wheel packages the C++ DSO as _C.so for python bindings.
We'd like the python wheel to have the C++ backend be standalone, so it can be extracted/used by C++ applications, like is done today for the PyTorch wheels.
This means:
- export DSO as
libtorchvision.soinstead of_C.so - do not hardlink
libtorchvision.soagainstlibtorch_python.so.- maybe
_C.sois kept for symbols that requirelibtorch_python.so?
- maybe
- export cpp headers
- export CMake configs
Motivation, pitch
C++ developers can currently use the distributed PyTorch wheels to develop C++ native applications against libtorch, as libraries, headers, and cmake configs are available in the wheels.
C++ developers who also need to use torchvision cannot leverage the standard vision wheel the same way even though all C++ symbols are available in _C.so. Instead, they must build libtorchvision C++ from source which is more cumbersome, requires extra dev packages to be installed, especially for cuda support.
Additional context
see ld links for torchvision 0.22.0+cu128 (wheel)
libc.so.6
libc10.so
libc10_cuda.so
libcudart.so.12
libdl.so.2
libgcc_s.so.1
libm.so.6
libpthread.so.0
librt.so.1
libstdc++.so.6
libtorch.so
libtorch_cpu.so
libtorch_cuda.so
libtorch_python.so # requires python
linux-vdso.so.1
see ld links for c++ source build of torchvision
no link against
libtorch_python.so
libc.so.6
libc10.so
libc10_cuda.so
libcudart.so.12
libdl.so.2
libgcc_s.so.1
libm.so.6
libpthread.so.0
librt.so.1
libstdc++.so.6
libtorch.so
libtorch_cpu.so
libtorch_cuda.so
linux-vdso.so.1
example of a cpp torchvision installation with files needed for C++ development
The install tree below can be imported for building with CMake with:
cmake ... -D TorchVision_ROOT="$torch_vision_install_dir" # Or add to CMAKE_PREFIX_PATH
find_package(TorchVision)
├── include
│ └── torchvision
│ ├── io
│ │ └── image
│ │ ├── cpu
│ │ │ ├── common_jpeg.cpp
│ │ │ ├── common_jpeg.h
│ │ │ ├── common_png.h
│ │ │ ├── decode_gif.cpp
│ │ │ ├── decode_gif.h
│ │ │ ├── decode_image.cpp
│ │ │ ├── decode_image.h
│ │ │ ├── decode_jpeg.cpp
│ │ │ ├── decode_jpeg.h
│ │ │ ├── decode_png.cpp
│ │ │ ├── decode_png.h
│ │ │ ├── encode_jpeg.cpp
│ │ │ ├── encode_jpeg.h
│ │ │ ├── encode_png.cpp
│ │ │ ├── encode_png.h
│ │ │ ├── exif.h
│ │ │ ├── giflib
│ │ │ │ ├── dgif_lib.c
│ │ │ │ ├── gif_hash.c
│ │ │ │ ├── gif_hash.h
│ │ │ │ ├── gif_lib.h
│ │ │ │ ├── gif_lib_private.h
│ │ │ │ ├── gifalloc.c
│ │ │ │ └── openbsd-reallocarray.c
│ │ │ ├── read_write_file.cpp
│ │ │ └── read_write_file.h
│ │ ├── cuda
│ │ │ ├── decode_jpeg_cuda.cpp
│ │ │ ├── encode_decode_jpegs_cuda.h
│ │ │ ├── encode_jpegs_cuda.cpp
│ │ │ └── encode_jpegs_cuda.h
│ │ ├── image.cpp
│ │ ├── image.h
│ │ └── image_read_mode.h
│ ├── macros.h
│ ├── ops
│ │ ├── autocast
│ │ │ ├── deform_conv2d_kernel.cpp
│ │ │ ├── nms_kernel.cpp
│ │ │ ├── ps_roi_align_kernel.cpp
│ │ │ ├── ps_roi_pool_kernel.cpp
│ │ │ ├── roi_align_kernel.cpp
│ │ │ └── roi_pool_kernel.cpp
│ │ ├── autograd
│ │ │ ├── deform_conv2d_kernel.cpp
│ │ │ ├── ps_roi_align_kernel.cpp
│ │ │ ├── ps_roi_pool_kernel.cpp
│ │ │ ├── roi_align_kernel.cpp
│ │ │ └── roi_pool_kernel.cpp
│ │ ├── cpu
│ │ │ ├── deform_conv2d_kernel.cpp
│ │ │ ├── nms_kernel.cpp
│ │ │ ├── ps_roi_align_kernel.cpp
│ │ │ ├── ps_roi_pool_kernel.cpp
│ │ │ ├── roi_align_common.h
│ │ │ ├── roi_align_kernel.cpp
│ │ │ └── roi_pool_kernel.cpp
│ │ ├── cuda
│ │ │ ├── cuda_helpers.h
│ │ │ ├── deform_conv2d_kernel.cu
│ │ │ ├── nms_kernel.cu
│ │ │ ├── ps_roi_align_kernel.cu
│ │ │ ├── ps_roi_pool_kernel.cu
│ │ │ ├── roi_align_kernel.cu
│ │ │ └── roi_pool_kernel.cu
│ │ ├── deform_conv2d.cpp
│ │ ├── deform_conv2d.h
│ │ ├── nms.cpp
│ │ ├── nms.h
│ │ ├── ops.h
│ │ ├── ps_roi_align.cpp
│ │ ├── ps_roi_align.h
│ │ ├── ps_roi_pool.cpp
│ │ ├── ps_roi_pool.h
│ │ ├── roi_align.cpp
│ │ ├── roi_align.h
│ │ ├── roi_pool.cpp
│ │ └── roi_pool.h
│ ├── vision.cpp
│ └── vision.h
├── lib
│ └── libtorchvision.so
└── share
└── cmake
└── TorchVision
├── TorchVisionConfig.cmake
├── TorchVisionConfigVersion.cmake
├── TorchVisionTargets-noconfig.cmake
└── TorchVisionTargets.cmake
How libtorch C++ can be used today with C++ CMake projects
torch_install=$(python3 -c "import torch; print(torch.__path__[0])")
cmake ... -D Torch_ROOT="$torch_install" # Or add to CMAKE_PREFIX_PATH
find_package(Torch)
> ls $torch_install/include
ATen/ dnnl_config.h dnnl_sycl_types.h fp16.h psimd.h tensorpipe/
c10/ dnnl_debug.h dnnl_threadpool.h fxdiv.h pthreadpool.h THC/
caffe2/ dnnl.h dnnl_types.h kineto/ pybind11/ torch/
clog.h dnnl_ocl.h dnnl_version.h libshm.h qnnpack_func.h xnnpack.h
cpuinfo.h dnnl_sycl.h experiments-config.h nnpack.h sleef.h
> ls $torch_install/include
ATen/ dnnl_config.h dnnl_sycl_types.h fp16.h psimd.h tensorpipe/
c10/ dnnl_debug.h dnnl_threadpool.h fxdiv.h pthreadpool.h THC/
caffe2/ dnnl.h dnnl_types.h kineto/ pybind11/ torch/
clog.h dnnl_ocl.h dnnl_version.h libshm.h qnnpack_func.h xnnpack.h
cpuinfo.h dnnl_sycl.h experiments-config.h nnpack.h sleef.h
> ls $torch_install/share/cmake/Torch/
TorchConfig.cmake TorchConfigVersion.cmake
> ls $torch_install/lib
libarm_compute_graph.so* libcudnn_ops.so.9* libnvpl_blas_lp64_gomp.so.0*
libarm_compute.so* libcudnn.so.9* libnvpl_lapack_core.so.0
libc10_cuda.so* libcufft.so.11* libnvpl_lapack_lp64_gomp.so.0
libc10.so* libcufile_rdma.so.1* libnvrtc-builtins.so.12.8*
libcaffe2_nvrtc.so* libcufile.so.0* libnvrtc.so.12*
libcublasLt.so.12* libcupti.so.12* libnvToolsExt.so.1*
libcublas.so.12* libcurand.so.10* libshm.so*
libcudart.so.12* libcusolver.so.11* libtorch_cpu.so*
libcudnn_adv.so.9* libcusparseLt.so.0 libtorch_cuda_linalg.so*
libcudnn_cnn.so.9* libcusparse.so.12* libtorch_cuda.so*
libcudnn_engines_precompiled.so.9* libgfortran.so.5* libtorch_global_deps.so*
libcudnn_engines_runtime_compiled.so.9* libgomp.so.1* libtorch_python.so*
libcudnn_graph.so.9* libnvJitLink.so.12* libtorch.so*
libcudnn_heuristic.so.9* libnvpl_blas_core.so.0*
This looks reasonable, please do not hesitate to propose a PR to enable that (it might already be the case for CMakeBuilds)
Hi folks, thanks for the feature request.
We have basic support for C++ extensions as described in https://github.com/pytorch/vision/tree/main/examples/cpp, but note that this isn't actively maintained / developed. It's not thoroughly tested, and there is no BC guarantee on the C++ APIs.
I'm afraid a more complete C++ API won't be in scope, however. The decision was made years ago to actively remove C++ APIs from torchvision, encouraging users to rely on torchscript (and now torch.export) for non-Python applications. Because of the significant overhead and maintenance cost that supporting a c++ backend would incur, I'm afraid I'm not able to commit to this at this point.
Thank you all
export the Python APIs via torchscript for non-python applications
@NicolasHug, could you guide me (and future lurkers) to the appropriate documentation?
This would be great, currently relying on OpenCV for image decoding and basic transformations before converting to tensors and would love to swap it out with torchvision. Unfortunately it seems like torchscript doesn't suffice for decoding so making the C++ API more user friendly would be really nice.
