Recommend a working conda installation sequence of torchvision, ffmpeg, opencv (both python and shared libraries + includes) together

[vadimkantorov]

Original context: https://github.com/pytorch/vision/issues/4240

Dfeault installation of torchvision and ffmpeg without version installs a non-workable ffmpeg 4.3 missing some libraries and gets installed for some reason from PyTorch channel.

I also read in other threads https://github.com/pytorch/vision/issues?q=is%3Aissue+is%3Aopen+ffmpeg that torchvision currently needs ffmpeg 4.2, but this is not reflected in dependency versions.

Also, building torchvision with ffmpeg from source is source of a lot of problems (in the issue search above)

Could README / torchvision:

1. recommend a known workable installation command of ffmpeg (maybe version-pinned? or with fixed dependency versions?)
2. recommend a known workable installation command of ffmpeg that lets you build torchvision from sources

At best it would be great to have a simple CI test installing from conda pytorch / torchvision / ffmpeg / opencv / maybe pillow-simd / open3d and a simple CI test for building from sources (as a regular user would).

@soumith

[bjuncek]

Hi @vadimkantorov

There are some issues regarding packaging FFMPEG directly with tochvision; last I recall it was licensing, but I'm trying to gather the info from people that have been working on it for the past years (specifically @andfoy and @fmassa ) and put all the information in the same issue. @vmoens proposed a solution in #4218 but that needs to be tested on different platforms.

[fmassa]

Hi Vadim,

Here is some clarification about the ffmpeg that gets pulled from the PyTorch channel:

• it installs a stripped out version of ffmpeg that doesn't contain GPL code
• it pins to a version that doesn't contain bug in FFmpeg that could lead to segfaults https://github.com/pytorch/vision/issues/2650#issuecomment-694915626
• we don't ship ffmpeg with torchvision (and don't make ffmpeg and explicit dependency either), because our video reading support is still in experimental mode.

It might be that in the meantime some dependencies got messed up with latest releases (which makes our ffmpeg package not workable anymore). We should revisit what we are doing and come up with a more robust solution

[vadimkantorov]

1. This should be clearly warned somehow, it's very non-obvious that PyTorch channels ships its own ffmpeg and that it takes precedence over regular ffmpeg package. Also, the shipped version's executable seems broken and misses some libraries. Maybe also ffmpeg could be compiled with torchvision banner so that it's clear that it torchvision's version + it may contain URL to docs. Users may also want to have a regular GPL ffmpeg installed side by side, so somehow there should be a recipe of having both simultaneously or this should be clearly discussed in the docs.

2. It still installed 4.3, while that issue says that it's 4.3 has the bug

3. Maybe you should not always ship it (for binary size?) or having an option of using dynamically loaded users-installed ffmpeg?

[fmassa]

@vadimkantorov yes, we definitely need to improve the situation here. Many of us were on holidays the past month (so things have been a bit slow), but we will be investigating and fixing it soon.

[vadimkantorov]

Given bugginess / unscrutinability of conda regarding conflicts, it would also be nice to have automated regular testing of installation recipes for conda, also including opencv which is also a frequent companion of torchvision. Some CI testing of installation commands was implemented within the scope of https://github.com/pytorch/pytorch/issues/62590 cc @malfet

[vadimkantorov]

Another instance is: https://github.com/pytorch/vision/issues/5616

It would be nice to have a nightly test of conda/pip installs of torchvision + opencv + ffmpeg -> this is a very common scenario, and even if it doesn't work, it'd be nice to have a "badge" indicating working or not working install. At the very least, there needs to exist known/tested conda+pip recipes of installation of these libraries...

[vadimkantorov]

Another idea: add open3d and have a fuzzer about import order: https://github.com/pytorch/pytorch/issues/74547. All these issues about which blas gets loaded and which OpenMP variant gets loaded...

[malfet]

cc: @janeyx99

[malfet]

We need to resolve this one before 1.12-rc1

[vadimkantorov]

another instance of non-existent tested recipe for building from source with ffmpeg: https://github.com/pytorch/vision/issues/5928

i propose various self-contained installation / building recipes are tested by CI and txt logs published somewhere -> then at least this is never a surprise that something doesn't work

cc @atalman