Behaviour changed on duration = 0 or N/A tracks : Is that intentional??

[crnkv]

Important Information

Provide following Information:

• tested on: mpv-x86_64-v3-20240517-git-47f60d1 (v0.38.0-256-g47f60d1c) / mpv-x86_64-v3-20240114-git-bd35dc8 (v0.37.0-161-gbd35dc8c) / mpv-x86_64-v3-20231231-git-abc2a74 (v0.37.0-135-gabc2a748) / mpv-lazy-2024V0 (v0.37.0-139-gab5b2503)
• Windows 10 22H2
• shinchiro's mpv-winbuild-cmake on github and sourceforge
• behaviour changed between v0.37.0-139-gab5b2503 and v0.37.0-161-gbd35dc8c (still there in v0.38.0-256-g47f60d1c)
• GPU: AMD, but irrelevant (using software decoding)
• Possible screenshot or video of visual glitches

Reproduction steps

• produce a video with one HEVC video stream and one AAC audio stream with a years-old all-in-one GUI tool which essentially executes a script D:\softwares\小丸工具箱rev236\tools\mp4box.exe -add "in.mp4#trackID=1:par=1:1:name=" -add "in.m4a:name=" -new "out.mp4" using an also years-old MP4Box of GPAC version 0.5.1-DEV-rev4929
• open with (different versions of) mpv, by dragging the file onto mpv.exe and by mpv.com -v "out.mp4"

Expected behavior

play the video

Actual behavior

mpv v0.37.0-139-gab5b2503 (and earlier) does play the video, while v0.37.0-161-gbd35dc8c (and later) simply skips (drops) all the frames and reports [cplayer] got EOF with no data before it then exits (mpv window closes immediately).

I'm well aware that the remuxed out.mp4 is broken in some sense, but the fact that mpv was once able to play it makes me wonder why mpv has dropped support for playing abnormal file (by ignoring 0 or N/A duration metadata).

(or DID mpv intentionally drop that "feature" in the first place (otherwise, an unwanted behaviour change)? Is there a reason why mpv just cannot ignore a track duration saying 0 or N/A? It seems to me that earlier versions of mpv followed some logic to decide ok mpv should play all frames in this specific case, but then this logic is discontinued in later versions.)

File details

D:\>ffprobe "out.mp4"
ffprobe version 7.0-essentials_build-www.gyan.dev Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 13.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isom
    creation_time   : 2023-08-09T17:09:48.000000Z
  Duration: 00:00:00.08, start: 0.000000, bitrate: 331445 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv, bt709), 720x1280 [SAR 1:1 DAR 9:16], 682 kb/s, 30 fps, 30 tbr, 16k tbn (default)
      Metadata:
        vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 149 kb/s (default)
      Metadata:
        vendor_id       : [0][0][0][0]

D:\>ffprobe -v trace "out.mp4" 2>&1 | findstr "edit"
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000019508ab4000] track[0].edit_count = 2
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000019508ab4000] Processing st: 0, edit list 0 - media time: -1, duration: 667
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000019508ab4000] Processing st: 0, edit list 1 - media time: 2128, duration: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000019508ab4000] track[1].edit_count = 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000019508ab4000] Processing st: 1, edit list 0 - media time: 0, duration: 0

Notice that Duration: 00:00:00.08 and duration: 0 are problematic metadata caused by that MP4Box remuxing, while at the same time, that old MP4Box reads Media Duration 00:00:32.966 and Track has 2 edit lists: track duration is 00:00:00.041, showing some sort of incompatibility (but yes of course, one should be now deprecated):

D:\softwares\小丸工具箱rev236\tools>MP4Box.exe -info "out.mp4"
* Movie Info *
        Timescale 600 - Duration 00:00:33.086
        2 track(s)
        Fragmented File: no
        File suitable for progressive download (moov before mdat)
        File Brand isom - version 1
        Created: GMT Wed Aug 09 17:09:48 2023
        Modified: GMT Wed Aug 09 17:09:48 2023

File has root IOD (9 bytes)
Scene PL 0xff - Graphics PL 0xff - OD PL 0xff
Visual PL: Not part of MPEG-4 Visual profiles (0xfe)
Audio PL: AAC Profile @ Level 2 (0x29)
No streams included in root OD

Track # 1 Info - TrackID 1 - TimeScale 16000 - Media Duration 00:00:32.966
Track has 2 edit lists: track duration is 00:00:00.041
Media Info: Language "Undetermined" - Type "vide:hev1" - 989 samples
Visual Track layout: x=0 y=0 width=720 height=1280
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x23
HEVC Video - Visual Size 720 x 1280
        HEVC Info: Profile IDC 1 - Level IDC 120 - Chroma Format 1
        NAL Unit length bits: 32 - general profile compatibility 0x60000000
        Parameter Sets: 1 VPS 1 SPS 1 PPS
        SPS resolution 720x1280 - Pixel Aspect Ratio 1:1 - Indicated track size 720 x 1280
        Bit Depth luma 8 - Chroma 8 - 1 temporal layers

Self-synchronized

Track # 2 Info - TrackID 2 - TimeScale 48000 - Media Duration 00:00:33.088
Track has 1 edit lists: track duration is 00:00:33.086
Media Info: Language "Undetermined" - Type "soun:mp4a" - 1551 samples
MPEG-4 Config: Audio Stream - ObjectTypeIndication 0x40
MPEG-4 Audio AAC LC - 2 Channel(s) - SampleRate 48000
Synchronized on stream 1
Alternate Group ID 1

By the way, I once suspected that the problem was FFmpeg being unable to handle multiple edit lists, but then I found two other remuxed files as counter-examples: One is produced by a browser js-based remuxer (of which I don't know the details, but I guess it contains an implementation of FFmpeg), which also contains 2 edit lists in visual track # 1 , but ffprobe reports normal duration time, and it is playable by all versions of mpv.

The other is produced by the previously said GUI MP4Box toolchain, but due to some unknown reasons (I deleted the original video/audio stream files long ago) the out2.mp4 has only 1 edit list in visual track # 1 , but ffprobe reports N/A or 0 duration time, and it is not playable by mpv v0.37.0-161-gbd35dc8c (and later).

D:\>ffprobe "out2.mp4"
... ...
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out2.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isom
    creation_time   : 2023-07-30T17:59:17.000000Z
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0[0x1](und): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 1001 kb/s, 60 fps, 60 tbr, 16k tbn (default)
      Metadata:
        vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 108 kb/s (default)
      Metadata:
        vendor_id       : [0][0][0][0]

D:\>ffprobe -v trace "out2.mp4" 2>&1 | findstr "edit"
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000023279094100] track[0].edit_count = 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000023279094100] Processing st: 0, edit list 0 - media time: 1072, duration: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000023279094100] track[1].edit_count = 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000023279094100] Processing st: 1, edit list 0 - media time: 0, duration: 0

Log file

mpv-output.txt ffprobe-20240518-165344.log mpv-v0.37.0-135-output.txt

Sample files

Sample files needed to reproduce this issue can be attached to the issue (preferred), or be uploaded to https://0x0.st/ or similar sites. (Only needed if the issue cannot be reproduced without it.) Do not use garbage like "cloud storage", especially not Google Drive.

[llyyr]

Does --demuxer-lavf-o=advanced_editlist=0 fix it?

Also please provide a sample file

[crnkv]

Does --demuxer-lavf-o=advanced_editlist=0 fix it?

Yes, this works. Tested on v0.38.0-256-g47f60d1c and v0.37.0-161-gbd35dc8c, and with a bunch of different files having the same problem.

[llyyr]

Then it's entirely coincidental that your files worked in the past. This is the commit that caused the change https://github.com/mpv-player/mpv/pull/13254

Disabling advanced editlists support breaks multiple files from FFmpeg's FATE suite, so I don't think reverting the commit is a good idea. Can you use ffmpeg or programs that don't produce broken files to mux your videos?

[crnkv]

Well but I mentioned before that there's another file out2.mp4 with only 1 edit list in visual track # 1, which is also not playable. Are you sure that this advanced editlists support still manifests in this case? Not a duration time issue?

edit: Oh wait... you mean, earlier versions just didn't see any metadata ever related to editlists (so it didn't see the duration 0 problem), not that mpv saw a duration = 0 metadata and decided to play it anyway?

[crnkv]

Then it's entirely coincidental that your files worked in the past. This is the commit that caused the change #13254

Disabling advanced editlists support breaks multiple files from FFmpeg's FATE suite, so I don't think reverting the commit is a good idea. Can you use ffmpeg or programs that don't produce broken files to mux your videos?

Can I ask one more question a bit off-topic but still related? I used the old MP4Box to reverse-extract (for compatibility concerns) the visual and audio streams, then used FFmpeg to remux them. The resulting video is fine, but I notice that FFmpeg automatically introduces edit lists in the resulting video. Well I didn't tell it to do so... Later, I found a muxer option -use_editlist 0 that can stop this. Although its default value is auto, but it seems that FFmpeg almost allways introduces edit lists without exception. Interestingly, I tried splitting visual/audio streams of normal mp4 files (which contain no edit lists) then remuxing them with FFmpeg. The default auto always leads to creating edit lists, despite that there's only one visual track and only one audio track, with no offset ever existed or demanded in cmdline options (i.e. no specific reason). So... is this editlist a feature encouraged by the community? Should I always use it (when muxing) even if there's no need? Otherwise why does that auto always create edit lists? P.S: I randomly checked some of my movies. Some mp4 do contain edit lists while some don't. That confuses me further more...