RenderHeads
[Android] After changing audio track playback freezes
Unity version
6000.1.14f1
Unity editor platform
macOS
AVPro Video edition
Ultra
AVPro Video version
v3.2.9
Device hardware
Pico4 Ultra
Which Android OS version are you using?
14
Unity Graphics API
OpenGLES 3
Video API
media3/ExoPlayer
Texture format
Y Cb Cr 420_OES
Audio output
System Direct
Any other Media Player component configuration required to reproduce the issue.
No response
Which output component(s) are you using?
Apply to Mesh
Any other component configuration required to reproduce the issue.
No response
The issue
Issue related to this question https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/2322
I'm trying to play a video with multiple audio tracks. After opening the video, I can get the list of tracks using:
var tracks = _mediaPlayer.AudioTracks.GetAudioTracks();
for (int i = 0; i < tracks.Count; i++)
{
DebugArea.Info( "Audio track {0}: {1} {2}", i, tracks[i].DisplayName, tracks[i].IsDefault);
}
Next I try to change the track using the code:
var tracks = _mediaPlayer.AudioTracks.GetAudioTracks();
_mediaPlayer.AudioTracks.SetActiveAudioTrack(tracks[0]);
And after that the video freezes. I can't play or pause or seek it anymore.
In additional i tried this video in native Pico4 video payer and all audio tracks work good.
And i see in logs:
09:37:04.470 7777 7777 E AVProVideo : ❌ Player_ExoPlayer.onPlayerError - onPlayerError - error: MediaCodecAudioRenderer error, index=5, format=Format(3, MVO, Jaskier, null, audio/ac3, null, -1, ru, [-1, -1, -1.0, null], [6, 48000]), format_supported=NO_UNSUPPORTED_TYPE
It looks like this codec is not supported. In this case, you need to be able to filter unsupported tracks
Media information
Here is video example: https://drive.google.com/file/d/18EYJ1oVkOrCID31UwC1fHRF3msHM10xx/view?usp=sharing
Logcat output
https://drive.google.com/file/d/1gaoAYNojeqZscTz71eEkQ5DaY6Y13tJF/view?usp=sharing
This will be fixed in the next release. Unsupported audio tracks will now be filtered out.
This will be fixed in the next release. Unsupported audio tracks will now be filtered out.
Thanks @MorrisRH ! Is there any idea when to expect the new release?
AVPro Video version 3.3.0 has been released. Please let us know if it has not fixed your issue.
AVPro Video version 3.3.0 has been released. Please let us know if it has not fixed your issue.
After updating to 3.3.0, I couldn't play any videos on Pico. I hear sound, but there's no video. Or sometimes it renders one frame and that's it. It works in UnityEditor. I'm trying to figure out what's going on.
hmm, that doesn't sound right. Are you able to provide a full, unfiltered logcat please? What specs are the videos? resolution/codecs/fps/bitrate etc?
hmm, that doesn't sound right. Are you able to provide a full, unfiltered logcat please? What specs are the videos? resolution/codecs/fps/bitrate etc?
I agree. That's why I'm trying to figure it out myself first. Maybe I updated something incorrectly.
@Chris-RH , Looks like something wrong with OES. I changed texture format to BGRA and shader (AVProVideo/VR/InsideSphere Unlit Transparent (stereo+color+fog+alpha)) and after that video starts work. I didn't have this problem before the AVPro update. I'm playing a video that definitely worked before the update.
With OES i use shader AVProVideo/VR/InsideSphere Unlit Transparent (stereo+color+alpha) - Android OES ONLY
Thanks for bringing this to our attention. We'll investigate this tomorrow :)
@Chris-RH , I've tried demo360 scene with Cones360Stereo-2K30-H264.mp4 video. The same issue with OES. I tested on Quest2 and Pico4Ultra.
We think we know what the cause is and are looking at how to resolve it :)
@Chris-RH , i've tried 3.3.1. Issue with OES was fixed. Video play without problem now.
But things aren't quite right with audio tracks yet. Now, if I open a video that previously froze due to an audio track, it doesn't freeze and plays without sound, which is good. But if I open a video with multiple audio tracks, the unsupported audio track still appears in the list, and switching to it causes the video to freeze.
For example here is video info:
Input #0, matroska,webm, from 'http://192.168.1.10:32469/object/8e58c5b6e27719034570/file.mkv':
Metadata:
title : ● My Neighbor Totoro ●
encoder : libebml v1.4.2 + libmatroska v1.6.4
creation_time : 2023-02-05T13:52:39.000000Z
Duration: 01:26:25.19, start: 0.000000, bitrate: 15315 kb/s
Stream #0:0(jpn): Video: h264 (High), yuv420p(progressive), 1920x1036 [SAR 1:1 DAR 480:259], 23.98 fps, 23.98 tbr, 1k tbn (default) (forced)
Metadata:
title : FortunaTV
BPS : 14284381
DURATION : 01:26:25.180000000
NUMBER_OF_FRAMES: 124320
NUMBER_OF_BYTES : 9258386434
_STATISTICS_WRITING_APP: mkvmerge v63.0.0 ('Everything') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2023-02-05 13:52:39
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:1(rus): Audio: aac (LC), 48000 Hz, stereo, fltp (default)
Metadata:
title : Русский (Reanimedia)
BPS : 193296
DURATION : 01:26:25.194000000
NUMBER_OF_FRAMES: 243056
NUMBER_OF_BYTES : 125284840
_STATISTICS_WRITING_APP: mkvmerge v63.0.0 ('Everything') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2023-02-05 13:52:39
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:2(rus): Audio: eac3, 48000 Hz, 5.1(side), fltp, 448 kb/s
Metadata:
title : Русский 5.1 (Reanimedia)
BPS : 448000
DURATION : 01:26:25.184000000
NUMBER_OF_FRAMES: 162037
NUMBER_OF_BYTES : 290370304
_STATISTICS_WRITING_APP: mkvmerge v63.0.0 ('Everything') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2023-02-05 13:52:39
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:3(rus): Audio: aac (LC), 48000 Hz, stereo, fltp
Metadata:
title : Русский (Сонотэк)
BPS : 192515
DURATION : 01:26:25.194000000
NUMBER_OF_FRAMES: 243056
NUMBER_OF_BYTES : 124779065
_STATISTICS_WRITING_APP: mkvmerge v63.0.0 ('Everything') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2023-02-05 13:52:39
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:4(jpn): Audio: aac (LC), 48000 Hz, stereo, fltp
Metadata:
title : Японский
BPS : 192282
DURATION : 01:26:25.194000000
NUMBER_OF_FRAMES: 243056
NUMBER_OF_BYTES : 124627791
_STATISTICS_WRITING_APP: mkvmerge v63.0.0 ('Everything') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2023-02-05 13:52:39
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:5(rus): Subtitle: ass (ssa)
Metadata:
title : Русские
BPS : 55
DURATION : 01:25:48.660000000
NUMBER_OF_FRAMES: 579
NUMBER_OF_BYTES : 35979
_STATISTICS_WRITING_APP: mkvmerge v63.0.0 ('Everything') 64-bit
_STATISTICS_WRITING_DATE_UTC: 2023-02-05 13:52:39
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:6: Video: png, rgb48be(pc, gbr/unknown/unknown), 600x600 [SAR 2835:2835 DAR 1:1], 90k tbr, 90k tbn (attached pic)
Metadata:
filename : cover.png
mimetype : image/png
In the list of tracks i see all audio tracks except of "Stream #0:4(jpn)". But when i try to select "Stream #0:4(jpn)" video is freeze and in logcat i see exception:
11:21:21.441 15801 20767 E ExoPlayerImplInternal: Playback error
11:21:21.441 15801 20767 E ExoPlayerImplInternal: F.T: MediaCodecAudioRenderer error, index=6, format=Format(3, Русский 5.1 (Reanimedia), video/x-matroska, audio/eac3, null, -1, ru, [-1, -1, -1.0, null], [6, 48000]), format_supported=NO_UNSUPPORTED_TYPE
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at F.i.W(Unknown Source:44)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at F.i.V(Unknown Source:1)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at Q.w.r1(Unknown Source:102)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at Q.w.I1(Unknown Source:3)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at Q.w.H0(Unknown Source:11)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at Q.w.x1(Unknown Source:246)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at H.T.x1(Unknown Source:10)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at Q.w.H1(Unknown Source:21)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at Q.w.g(Unknown Source:34)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at F.F1.I(Unknown Source:10)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at F.X0.C(Unknown Source:104)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at F.X0.handleMessage(Unknown Source:379)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at android.os.Handler.dispatchMessage(Handler.java:102)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at android.os.Looper.loopOnce(Looper.java:209)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at android.os.Looper.loop(Looper.java:303)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at android.os.HandlerThread.run(HandlerThread.java:67)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: Caused by: Q.w$c: Decoder init failed: [-49999], Format(3, Русский 5.1 (Reanimedia), video/x-matroska, audio/eac3, null, -1, ru, [-1, -1, -1.0, null], [6, 48000])
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at Q.w.s1(Unknown Source:181)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: at Q.w.r1(Unknown Source:82)
11:21:21.441 15801 20767 E ExoPlayerImplInternal: ... 13 more
Full logcat: https://drive.google.com/file/d/1L9Rc4V4jdRfprfV5otTIfs2jIOeBIOgn/view?usp=sharing