AV1 support

[peters]

@alalek FFMPEG 4 now supports libaom enabling encoding of AV1 videos. I'm thinking of adding support for it in opencv_thirdparty. If the final binary opencv_ffmpeg.dll does not increase that much in size would this be something that could considered to be included in the official build?

[alalek]

What is license of libaom? (we can't build with other GPL libraries; LGPL is fine)

It would be nice to have AV1 support. +1-2Mb size increase should not be a huge issue.

[peters]

@alalek Great! Wikipedia says it's a FreeBSD license. Link to libaom license: https://aomedia.googlesource.com/aom/+/dev/LICENSE

[peters]

make_mingw.sh

# Libaom
libaom_DIR=${BUILD_DIR}/libaom
libaom_x86_DIR=${BUILD_DIR}/libaom_x86
libaom_x64_DIR=${BUILD_DIR}/libaom_x64
libaom_configure_OPTIONS="-DENABLE_EXAMPLES=OFF -DENABLE_TOOLS=OFF -DCONFIG_UNIT_TESTS=0 -DCONFIG_MULTITHREAD=0"
if [ ! -d ${libaom_DIR} ]; then
  echo "Libaom source tree is not found"
  exit 1
fi
##[ -d ${libaom_x86_DIR} ] ||
#(
cd ${libaom_DIR}
mkdir -p ${libaom_x86_DIR} ${libaom_x86_DIR}/build_output
rsync -a ./ ${libaom_x86_DIR} --exclude .git
cd ${libaom_x86_DIR}/build_output
cmake -DAOM_TARGET_CPU=generic -DCMAKE_TOOLCHAIN_FILE=${libaom_x86_DIR}/build/cmake/toolchains/x86-mingw-gcc.cmake -DCMAKE_INSTALL_PREFIX=${libaom_x86_DIR}/install ${libaom_configure_OPTIONS}  ..
make -j ${CPU_COUNT}
make install
#)
#[ -d ${libaom_x64_DIR} ] ||
(
cd ${libaom_DIR}
mkdir -p ${libaom_x64_DIR} ${libaom_x64_DIR}/build_output
rsync -a ./ ${libaom_x64_DIR} --exclude .git
cd ${libaom_x64_DIR}/build_output
cmake -DAOM_TARGET_CPU=generic -DCMAKE_TOOLCHAIN_FILE=${libaom_x64_DIR}/build/cmake/toolchains/x86_64-mingw-gcc.cmake -DCMAKE_INSTALL_PREFIX=${libaom_x64_DIR}/install ${libaom_configure_OPTIONS}  ..
make -j ${CPU_COUNT}
make install
)

Add --enable-libaom to FFMPEG_CONFIGURE_OPTIONS

Add ffmpeg link directory x86: -L${libaom_x64_DIR}/install/lib Add ffmpeg link directory x64: -L${libaom_x86_DIR}/install/lib Add link library x86/x64: bcrypt https://patchwork.ffmpeg.org/patch/8243/

ffmpeg_version.cmake

set(FFMPEG_libpostproc_FOUND 1)
set(FFMPEG_libavcodec_FOUND 1)
set(FFMPEG_libavformat_FOUND 1)
set(FFMPEG_libavutil_FOUND 1)
set(FFMPEG_libswscale_FOUND 1)
set(FFMPEG_libavresample_FOUND 1)

set(FFMPEG_libpostproc_VERSION 55.1.100)
set(FFMPEG_libavcodec_VERSION 58.18.100)
set(FFMPEG_libavformat_VERSION 58.12.100)
set(FFMPEG_libavutil_VERSION 56.14.100)
set(FFMPEG_libswscale_VERSION 5.1.100)
set(FFMPEG_libavresample_VERSION 4.0.0)

download_src.sh

Update ffmpeg to tag n4.0 NB! FFmpeg has dropped support for Windows XP. Add the following line: update libaom https://aomedia.googlesource.com/aom dev

Probably should wait until they have tagged a new release from master.

Winning

/cc @alalek I have a local build working now and AV1 encoding/decoding is working fine. You should probably wait to include AV1 support until they tag a new version from dev branch though. Just wanted to give you the recipe when you have time to update ffmpeg binaries :)

[peters]

Binary size for ffmpeg (n4.0) x64 build is roughly 24mb libaom included.

[alalek]

Thank you! Could you please open PR to ffmpeg/master? I will check it and necessary updates.

[alalek]

Great news: libaom 1.0 release

[peters]

@alalek I'm upgrading to v1.0.0 now. Going to run it in our staging environment for a few days. If everything works out I'll submit a PR.

We should probably also consider adding support for their new image format https://aomediacodec.github.io/av1-avif/.

[peters]

@alalek https://github.com/xiph/rav1e support just landed in https://github.com/FFmpeg/FFmpeg/commit/d8bf24459b694338de4ceb2a2e6d4d2949d6658d. Once available in the next release of ffmpeg I will submit a PR.

[asmorkalov]

@peters Do you plan to work on the patch in mean time?

[peters]

@asmorkalov Unfortunately, no.

[peters]

Alternative encoder/decoder to rav1e: https://github.com/OpenVisualCloud/SVT-AV1

[pwuertz]

What is the preferred path forward for this? FFMPEG seems to support various backends now, including rav1e and dav1d. Does OpenCV automatically gain AV1 support if built against an AV1 enabled FFMPEG?

[peters]

@pwuertz In my opinion the best way forward is to modify the docker build script in opencv_3rdparty

Does OpenCV automatically gain AV1 support if built against an AV1 enabled FFMPEG?

That is correct.

@alalek Maybe you could provide @pwuertz some guidance? :) Maybe we should consider using github actions as it supports build matrixes.

[peters]

AV1 encoding support using rav1e just landed in ffmpeg 4.3 release: https://github.com/FFmpeg/FFmpeg/blob/master/Changelog#L28

[peters]

libaom v2.0.0 has been released: https://aomedia.googlesource.com/aom/+/refs/tags/v2.0.0/CHANGELOG.

[peters]

I have a working build for rav1e based on this workflow. The problem is that it will add about 34 MB to the ffmpeg wrapper. libaom adds around 7 MB. @alalek What do you think? If you want I could submit a PR that uses GitHub Actions to the https://github.com/opencv/opencv_3rdparty repository. Then we could attach the rav1e enabled ffmpeg wrapper to the GitHub Release and consumers could choose which binary to download during build via CMAKE flag?

There is also the case of decoding support. The library that looks the most promising right now is https://github.com/videolan/dav1d. I have not tried building it yet because we do not require decoding support atm. But if we choose to enable decoding support in the future, the build pipeline should IMHO be improved upon :)

[alalek]

@peters Thank you for updates! Yes, feel free to create PR with necessary changes. Meantime we will discuss how to properly support multiple FFmpeg wrapper (or still keep single one by default with providing build flag)

[Tsubajashi]

is there any news regarding this? AV1 Support would be awesome!

[piwawa]

Did AV1 support added? I download a '.mkv' video from youtube, when i use ''cap = cv2.VideoCapture('2.mkv')" it errored, as follows: [av1 @ 0000028a88053c40] Your platform doesn't suppport hardware accelerated AV1 decoding. [av1 @ 0000028a88053c40] Failed to get pixel format.