[BUG]: AV1 is not available

[DenizUgur]

Package version

2.4.0-exp.8

Environment

* OS: MacOS
* Unity version: 2021.3.6f1

Steps To Reproduce

I'm using the receiver sample demo and I don't seem to be able to activate AV1 codec. What are the requirements for enabling AV1 stream?

Current Behavior

AV1 does not start the stream

Expected Behavior

AV1 should start the stream

Anything else?

No response

[karasusan]

Have you tried the package sample? https://docs.unity3d.com/Packages/[email protected]/manual/sample.html

[DenizUgur]

I was trying the sample from renderstreaming package. The one where we were able to select codecs from a dropdown. Now that seems to be gone or changed so I can't test it anymore. Also I've imported this package's sample into a new project but there are some errors so I'm not sure how to properly test AV1. My end goal is to use AV1 in place of VP8 on renderstreaming samples

[kannan-xiao4]

@DenizUgur What is the combination of sender and receiver? (Unity-Browser?)

[DenizUgur]

Here is my setup:

• Editor: 2021.3.6f1
• Unity Render Streaming: 3.1.0-exp.3
• Chrome: 104.0.5112.101
• macOS: 12.5

I've verified that player uses VP8 by chrome://webrtc-internals/. There used to be an option to select which codec I want to use but now that seems to be gone.

[karasusan]

@DenizUgur The feature of codec configuration is currently implemented in the develop branch of Unity Render Streaming repository.

[DenizUgur]

Okay, I have changed to develop branch and tested with the same configuration but I'm not seeing any video right now.

[karasusan]

@DenizUgur To fix this issue, we have to update the libwebrtc. The cause of issue is the mismatch of mimetype of AV1 because the old libwebrtc defines video/AV1X as the mimetype of AV1. We already have a plan to update the library near future.

[karasusan]

memo: WRS-97

[DenizUgur]

I'm currently modifying this repository in downstream so if it's just a matter of modifying the WEBRTC_VERSION parameter in build script related to libwebrtc then to which version should I change it?

[karasusan]

@DenizUgur changeset is here. But we need to fix the code because some APIs in libwebrtc are changed. https://source.chromium.org/chromium/_/webrtc/src.git/+/6b19d8273bb718833b562def87f92b9f0becb39e

[DenizUgur]

Is it possible to upgrade NVENC to 12.0 to support AV1 there?

https://developer.nvidia.com/blog/av1-encoding-and-fruc-video-performance-boosts-and-higher-fidelity-on-the-nvidia-ada-architecture/

[karasusan]

@DenizUgur Wow, it is good news for us. Thank for the heads-up.

[DenizUgur]

No problem 👍

@karasusan Oh and is there an ETA on when you guys will update the libwebrtc? If it's soon then I won't do it myself.

[karasusan]

@DenizUgur We don't have the plan for the update about NvCodec in this year. We are focusing on improving the stability for a while.

[DenizUgur]

Hi @karasusan, I was talking about updating libwebrtc for AV1 compatibility.

@DenizUgur changeset is here. But we need to fix the code because some APIs in libwebrtc are changed. https://source.chromium.org/chromium/_/webrtc/src.git/+/6b19d8273bb718833b562def87f92b9f0becb39e

[karasusan]

@DenizUgur We are going to start updating libwebrtc on next Jan or Feb.

[DenizUgur]

I see, thank you.

[karasusan]

Fixed. https://github.com/Unity-Technologies/com.unity.webrtc/pull/843

[karasusan]

@DenizUgur We released the new version of the package last week. This issue is fixed. Please try it and let me know if you have other issues. Thank you.