olive
olive copied to clipboard
olive-editor
[BUILD] Improve dependency management
- Gather list of dependencies
- direct and transitive (deps of deps)
- required, optional, desirable, ... ? (what features such as codecs to we want to support?)
- min/max supported versions
- Create support matrix of OSes, compilers etc.
- Make a list of mandatory and useful tools for development
- Assess package manager options
Dependencies
-
Qt 5.9+ (also see https://wiki.qt.io/Building_Qt_5_from_Git)
- Git >= 1.6.x
- Perl >= 5.14 (for build system)
- Clang / LLVM >= 6.x (optional, for documentation)
- SSL (optional)
- jom (or nmake)
- Ubuntu/Debian/Mint:
- build-essential
- '^libxcb.*-dev'
- libx11-xcb-dev
- libglu1-mesa-dev
- libxrender-dev
- libxi-dev
- libxkbcommon-dev
- libxkbcommon-x11-dev
- libxcb-xinerama0-dev?
- build-dep qt5-default? (for automatic installation of dependencies)
- Qt WebKit:
- flex
- bison
- gperf
- libicu-dev
- libxslt-dev
- ruby
- CentOS 7:
- GCC 7+
- libxcb
- libxcb-devel
- xcb-util
- xcb-util-devel
- mesa-libGL-devel
- libxkbcommon-devel
- macOS:
- Xcode
- Windows:
- ANGLE
- Windows 10 SDK (with debug tools. Use standalone installer, not Visual Studio Installer or it will miss features!)
- Visual Studio 2017 or 2019 (or MinGW with g++ 7.3 or higher)
- ANGLE
-
FFmpeg 3.0+
- NASM
- YASM
- x264 (libx264)
- x265 (libx265)
- Ogg (libogg)
- Opus (libopus)
- Vorbis (libvorbis)
- Theora (libtheora)
- VPx (libvpx)
- WebP (libwebp)
- LAME (libmp3lame)
- Xvid (xvidcore)
- OpenJPEG
- libpng
- ...
-
OpenColorIO 2.x
- (installs most external packages automatically, but what about the Conan package?)
- expat
- IlmBase
- pybind11
- pystring
- OpenImageIO (optional)
- (from existing Conan recipe for opencolorio/1.1.1:)
- lcms/2.11
- yaml-cpp/0.6.3
-
OpenImageIO 1.6+
- Boost
- libtiff
- libpng
- libturbojpeg
- (from existing Conan recipe for openimageio/2.2.7.0:)
- openexr/2.5.2
- zlib/1.2.11
- libtiff/4.1.0
- jbig/20160605
- libjpeg/9d
- zlib/1.2.11
- xz_utils/5.2.4
- zstd/1.4.5
- libwebp/1.1.0
- fmt/7.0.3
- boost/1.74.0
- zlib/1.2.11
- bzip2/1.0.8
- tsl-robin-map/0.6.3
- pugixml/1.10
- libsquish/1.15
- libpng/1.6.37
- zlib/1.2.11
- libwebp/1.1.0
- openjpeg/2.3.1
- giflib/5.2.1
- freetype/2.10.2
- brotli/1.0.9
- bzip2/1.0.8
- libpng/1.6.37
- zlib/1.2.11
- opencolorio/1.1.1
- lcms/2.11
- yaml-cpp/0.6.3
- libjpeg/9d
- dcmtk (optional)
- libraw (optional)
-
OpenEXR (openexr/2.5.3)
- zlib/1.2.1
-
OpenTimelineIO
- CMake (for C++ development)
- Python (for Python development)
- pip
- PySide2
- pip
-
Crashpad
- depot_tools
- Python (2?)
-
Breakpad
- depot_tools
- autoconf / make?
Toolchains
Compilers
Windows:
- Visual Studio 15 2017 (cl.exe)
- Visual Studio 16 2019 (cl.exe)
Linux:
- GCC (g++)
- 9.3.1 in CI
- Clang (clang++)
- 10.0.0 in CI
- 11.x known to have issues
macOS:
- XCode / Clang (?)
Linkers
- Visual Studio 15 2017 (link.exe)
- Visual Studio 16 2019 (link.exe)
- bfd, gold, lld?
Orchestrators
- Ninja
Vcpkg still no support for multiple versions
https://devblogs.microsoft.com/cppblog/take-control-of-your-vcpkg-dependencies-with-versioning-support/
@itsmattkc Out of curiosity: What does vcpkg lack at the moment, that you decided to write you own dependency system? (https://github.com/olive-editor/dependencies)
Primarily it was to experiment with having full control and ensuring dependency compiling is a separate step from compiling Olive. I'm aware you can give vcpkg and brew custom scripts (and potentially use some form of caching to limit dependency compilation), and it's quite likely we'll port the scripts in that repo to such an approach at some point, but yeah it was mainly to experiment with a fully manual approach.
Well, too little, too late? It took 3 years(?) to get versioning support in this package manager(!), and I'm honestly surprised that it landed at all after months of silent postponing and basically no activity on the RFC. It makes me wonder how much commitment there is from Microsoft.
There doesn't seem to be a plan for a central public registry or support in GitHub Packages to serve prebuilt binaries, and I wouldn't want to maintain a custom registry, so it only solves half of the problem. Conan has a huge advantage here with Conan Center.
There are currently no vcpkg ports for OCIOv2 and OTIO, and Crashpad doesn't seem to be supported under Linux, so it's more or less on par with Conan in that regard. Not sure how good vcpkg's Linux and macOS support is meanwhile, but the catalogs used to be much smaller for them compared to the Windows one. Rez tries to solve problems that Olive doesn't really have and lacks in Windows support, so it's pretty much out of the picture.
Personally, I find Conan conceptually simpler and more user-friendly compared to the competition. Since there are no ports for core components such as OCIOv2 for either, the easiest solution is to compile them directly. That is actually a lot simpler (except for packaging) and means that one less tool needs to be understood and maintained. But I'm sure that the package managers and their ecosystems will evolve and deserve another look. I was kinda hoping that the ASWF would work on Conan ports, but they seem to be waiting on something that ILM works on (but will likely be Linux-only).
Not sure how good vcpkg's Linux and macOS support is meanwhile
Unfortunately, it seems that shared libraries are still not supported for Linux and macOS officially.
But I'm sure that the package managers and their ecosystems will evolve and deserve another look.
Yeah, I think they will all converge and provide the same features (taking inspiration from Rust's Cargo) at the end.
@itsmattkc Am I missing something or is your dependency system entirely dependent on GitHub Actions and thus can't be run locally?
You could more or less execute the commands one by one in your local shell I suppose, but there's also a project which tries to allow running GitHub Actions workflows locally: https://github.com/nektos/act
I don't think there's much point to keeping this issue open. One day we may implement a better system on the CI, but what we have is sufficient for now.