audio

audio copied to clipboard

Hi, after installing PyTorch 1.9.0, I tried to install torchaudio v0.9.0 (release/v0.9) from source on a ppc64le machine. However, I received an error that CMake is unable to locate protobuf. The server shows a CMake version of 3.11.4 and a ninja version of 1.10.1, and switching CMake to a higher version of 3.18.4 shows the same error.

CMake Warning at /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Caffe2/public/protobuf.cmake:88 (message):
  Protobuf cannot be found.  Depending on whether you are building Caffe2 or
  a Caffe2 dependent library, the next warning / error will give you more
  info.
Call Stack (most recent call first):
  /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:56 (include)
  /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:67 (find_package)


CMake Error at /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:58 (message):
  Your installed Caffe2 version uses protobuf but the protobuf library cannot
  be found.  Did you accidentally remove it, or have you set the right
  CMAKE_PREFIX_PATH? If you do not have protobuf, you will need to install
  protobuf and set the library path accordingly.
Call Stack (most recent call first):
  /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:67 (find_package)

but pip install protobuf gives

Requirement already satisfied: protobuf in /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages (3.14.0)
Requirement already satisfied: six>=1.9 in /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages (from protobuf) (1.15.0)

I saw the same error in (https://github.com/pytorch/audio/issues/1796) but as they installed another version of torchaudio and pytorch in the end, the issue above possibly remained for higher versions on the same system.

Here is the environment info pulled with https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py

PyTorch version: 1.9.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Red Hat Enterprise Linux release 8.3 (Ootpa) (ppc64le)
GCC version: (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5)
Clang version: Could not collect
CMake version: version 3.11.4
Libc version: glibc-2.17

Python version: 3.7.11 (default, Jul 27 2021, 15:02:35)  [GCC 7.3.0] (64-bit runtime)
Python platform: Linux-4.18.0-240.el8.ppc64le-ppc64le-with-redhat-8.3-Ootpa
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] torch==1.9.0
[pip3] torchtext==0.10.0a0+4da1de3
[pip3] torchvision==0.10.0
[conda] _pytorch_select           2.0                      cuda_2    https://opence.mit.edu
[conda] cudatoolkit               10.2.89            684.g752c550    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
[conda] numpy                     1.19.2           py37h6163131_0  
[conda] numpy-base                1.19.2           py37h75fe3a5_0  
[conda] pytorch                   1.9.0           cuda10.2_py37_1    https://opence.mit.edu
[conda] pytorch-base              1.9.0           h1234567_cuda10.2_py37_pb3.14_1    https://opence.mit.edu
[conda] torchtext                 0.10.0                   py37_1    https://opence.mit.edu
[conda] torchvision-base          0.10.0          cuda10.2_py37_1    https://opence.mit.edu

TIA for the help!

Hi @JeromeNi

The error suggests that there is an issue with the installation of PyTorch. How did you install PyTorch?

Looking at the following, it seems that pytorch-base package, which I assume is managed by your system administrator, is compiled with CUDA10.2 and protobuf 3.14.1 on Python 3.7.

[conda] pytorch                   1.9.0           cuda10.2_py37_1    https://opence.mit.edu
[conda] pytorch-base              1.9.0           h1234567_cuda10.2_py37_pb3.14_1    https://opence.mit.edu

Whereas the version you showed is 3.14.0.

Requirement already satisfied: protobuf in /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages (3.14.0)

So I imagine that using the same version of protobuf that is managed by the same administrator might help it.

Also I do not know the difference between pytorch-base and pytorch above, but if both of them are valid package, then having two packages is a possible source of error.

I found that protobuf 3.14.0 is likely the correct version, as there isn't a version 3.14.1 for protobuf on its GitHub.

I will ask the system admin to see why there are two pytorches. I'm not sure if this is helpful to figuring out the error, but I found that I could complie torchaudio v0.7.2 in my PyTorch 1.9.0 environment (as well as another PyTorch v1.7.1 environment; also with two pytorches). It's just that torchaudio v0.9.0 (with cmake) keeps reporting this protobuf error above...

@JeromeNi have you made any progress on this?