pytorch
Installing torchvision for ppc64le IBM architecture
I am in a weird scenario were I am forced to use torch 1.3.1 (due to hardware see: https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/). I read from the pytorch docs that it's corresponding version of torchvision is 0.4.1 (https://pypi.org/project/torchvision/):
Installation
We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.
Installation We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions. torch torchvision python master / nightly master / nightly >=3.6 1.5.0 0.6.0 >=3.5 1.4.0 0.5.0 ==2.7, >=3.5, <=3.8 1.3.1 0.4.2 ==2.7, >=3.5, <=3.7 1.3.0 0.4.1 ==2.7, >=3.5, <=3.7 1.2.0 0.4.0 ==2.7, >=3.5, <=3.7 1.1.0 0.3.0 ==2.7, >=3.5, <=3.7 <=1.0.1 0.2.2 ==2.7, >=3.5, <=3.7
but for some reason I have the wrong version of it:
torchvision 0.2.2 pypi_0 pypi
is there a way to install the right version of torchvision?
What I've tried:
First I tried force installing the right version with conda. Conda couldn't find the version of torchvision that I need:
$ conda install torchvision==0.4.2
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
PackagesNotFoundError: The following packages are not available from current channels:
- torchvision==0.4.2
Current channels:
- https://repo.anaconda.com/pkgs/main/linux-ppc64le
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/linux-ppc64le
- https://repo.anaconda.com/pkgs/r/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Then I proceeded to try to install it regardless with pip
$ pip install torchvision==0.4.2
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement torchvision==0.4.2 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3)
ERROR: No matching distribution found for torchvision==0.4.2
got an error too.
Is there anything else to try?
I tried but it failed:
conda install torchvision==0.4.2 -c pytorch
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
PackagesNotFoundError: The following packages are not available from current channels:
- torchvision==0.4.2
Current channels:
- https://conda.anaconda.org/pytorch/linux-ppc64le
- https://conda.anaconda.org/pytorch/noarch
- https://repo.anaconda.com/pkgs/main/linux-ppc64le
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/linux-ppc64le
- https://repo.anaconda.com/pkgs/r/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
related:
- crossposted SO: https://discuss.pytorch.org/t/force-installing-torchvision/97279
- crossposted reddit pytorch: https://www.reddit.com/r/pytorch/comments/iyf2qn/force_installing_torchvision/
- crossposted reddit ibm: https://www.reddit.com/r/IBM/comments/iyhzex/force_installing_torchvision_042_when_i_am_forced/
- real problem is installing torchmeta: https://github.com/tristandeleu/pytorch-meta/issues/95
- https://www.ibm.com/mysupport/s/forumsquestion?id=0D50z00006gaxV9CAI
- quora: https://www.quora.com/unanswered/How-does-one-install-specific-Python-packages-in-Conda-from-IBM-architectures
- reddit ibm2: https://www.reddit.com/r/newIBM/comments/iyij10/force_installing_torchvision_042_when_i_am_forced/
- gitissue for ibm: https://github.com/IBM/powerai/issues/268
cc @fmassa @vfdev-5
The torchvision issue tracker might be a better place for this: https://github.com/pytorch/vision/issues
@brando90 interesting issue. I just tried in a clean conda env (x86 linux) to install both pytorch and torchvision :
conda install pytorch=1.3.1 torchvision -c pytorch
and it proposes to install the correct version:
The following packages will be downloaded:
package | build
---------------------------|-----------------
cffi-1.14.3 | py36he30daa8_0 222 KB
intel-openmp-2020.2 | 254 786 KB
mkl-2020.2 | 256 138.3 MB
mkl_fft-1.2.0 | py36h23d657b_0 149 KB
ninja-1.10.1 | py36hfd86e86_0 1.4 MB
numpy-1.19.1 | py36hbc911f0_0 21 KB
numpy-base-1.19.1 | py36hfa32c7d_0 4.1 MB
olefile-0.46 | py36_0 48 KB
pillow-7.2.0 | py36hb39fc2d_0 619 KB
pycparser-2.20 | py_2 94 KB
pytorch-1.3.1 |py3.6_cuda10.1.243_cudnn7.6.3_0 428.0 MB pytorch
torchvision-0.4.2 | py36_cu101 14.8 MB pytorch
Can you try to run the command with pytorch=1.3.1 which normally shouldn't reinstall existing pytorch ?
EDIT: Probably, the issue is with linux-ppc64le architecture and the only noarch binary is 0.2.2 according to anaconda: https://anaconda.org/pytorch/torchvision
Can you try to build it from sources, from github branch v0.4.2 : https://github.com/pytorch/vision/tree/v0.4.2 ?
@vfdev-5 Your hunch is correct. As of torchvision==0.3 the provided binaries are precompiled for a few selected platforms. Thus, if you try to install with an unsupported platform it falls back to the last version without platform requirements, i. e. torchvision==0.2.2.
In order to install the correct version of torchvision on PPC64le I found it easy to just install the IBM MLDL framework which includes correct versions of pytorch and torchvision-base:
conda install powerai
PackagesNotFoundError: The following packages are not available from current channels: powerai
