com.unity.webrtc icon indicating copy to clipboard operation
com.unity.webrtc copied to clipboard

Published 20 hours ago • verified publisher icon Unity-Technologies

[BUG]: Video resolution with odd numbered size is resized unintendedly

Open mgodfreymodest opened this issue 2 years ago • 12 comments

Package version

2.4.0-exp.7

Environment

* OS:Windows 10
* Unity version:2019.4

Steps To Reproduce

  1. establish a stream between android and Windows, including video
  2. call ReplaceTrack on the video track on Android, including a size change
  3. observe on Windows that UnityVideoRenderer.OnVideoFrameResize does not get called on Windows

Current Behavior

OnVideoFrameResize doesn't get called, but only on Windows. ReplaceTrack with a new video track of the same size continues to stream. Changing the video stream on Windows, the OnVideoFrameResize call happens on Android.

Expected Behavior

Windows should also receive the callback from the c++ side.

Anything else?

This is a new issue, not found in 2.4.0-exp6.

mgodfreymodest avatar May 18 '22 16:05 mgodfreymodest

memo: WRS-328

karasusan avatar May 19 '22 01:05 karasusan

More details: A texture of size 720x405 does not trigger the callback. A texture of size 720x400 does trigger the callback.

Do the dimensions need to be a multiple of some number?

mgodfreymodest avatar May 30 '22 17:05 mgodfreymodest

@mgodfreymodest Thank you for details. That is weird... Do you know what codec you are using?

karasusan avatar May 31 '22 00:05 karasusan

I'm not sure exactly how to get the codec information, but the documentation suggests looking at the RTCRtpSender.

On Android I get the following mime types out of the sender codecs field: video/VP8, video/H264, video/VP9 and video/rtx

On Windows I get the following mime types out of the sender codecs field: video/VP8, video/H264, video/VP9 and video/rtx

Is this useful information?

mgodfreymodest avatar May 31 '22 11:05 mgodfreymodest

@mgodfreymodest VP8 codec on windows. it works. H264 works too. If you check it, please checkout the repository and try the develop branch.

https://user-images.githubusercontent.com/1132081/171544759-347e1b47-e312-42f1-8040-0482451ff035.mp4

karasusan avatar Jun 02 '22 03:06 karasusan

Using exp7 with the develop's sample scenes, my h264 codecs do a mix of changing the resolution from 720x405 on the send side to 720x406 on the received video, and crashing unity when you replace tracks to a differently-sized video.

There are several h264 options, in the following format, all behaving this way: Screenshot 2022-06-02 103506

mgodfreymodest avatar Jun 02 '22 13:06 mgodfreymodest

@mgodfreymodest Sure, I think the crash is fixed by this PR #737. Could you test the develop branch?

karasusan avatar Jun 07 '22 02:06 karasusan

I installed the plugin dll's from the develop branch. The sizing issue persists in my app.

When testing with the ReplaceTrack sample, still getting crashes when replacing from 720x400 to 720x405 using the h264 codecs.

mgodfreymodest avatar Jun 07 '22 11:06 mgodfreymodest

@mgodfreymodest Ah, I haven't uploaded the latest plugin. I am checking

karasusan avatar Jun 08 '22 02:06 karasusan

@mgodfreymodest Updated. Please try it again. https://github.com/Unity-Technologies/com.unity.webrtc/pull/742

karasusan avatar Jun 09 '22 02:06 karasusan

Using the latest plugin version on the sample code, the crash is resolved for h264.

Trying the latest plugin in my app, only even-numbered sizes stream, still.

mgodfreymodest avatar Jun 09 '22 10:06 mgodfreymodest

@mgodfreymodest Sure, this is a issue that the odd numbered size is resized unintendedly.

karasusan avatar Jun 14 '22 02:06 karasusan