mpv
mpv copied to clipboard
mpv-player
ASS subs are not rendered correctly in gpu-next
mpv Information
mpv v0.38.0-525-g265056fa Copyright © 2000-2024 mpv/MPlayer/mplayer2 projects
built on Jun 23 2024 00:07:06
libplacebo version: v7.349.0 (v7.349.0-rc1-3-g1fd3c7b-dirty)
FFmpeg version: N-115975-g0c0e7ec81
FFmpeg library versions:
libavcodec 61.8.100
libavdevice 61.2.100
libavfilter 10.2.102
libavformat 61.3.104
libavutil 59.25.100
libswresample 5.2.100
libswscale 8.2.100
Other Information
- Windows version: Microsoft Windows [Version 10.0.19045.4291]
- GPU model, driver and version: RTX 3080, 551.86
- Source of mpv: shinchiro latest windows
- Introduced in version: Unknown
Reproduction Steps
Load provided video with gpu-next. --no-config --vo=gpu-next
Expected Behavior
Subs are displayed like in normal rending.
Actual Behavior
Subs are displayed incorrectly, looks like some frames are dropped or the layers are displayed in the wrong order. Instead of being white with a purple border they are just purple.
Log File
Sample Files
I carefully read all instruction and confirm that I did the following:
- [X] I tested with the latest mpv version to validate that the issue is not already fixed.
- [X] I provided all required information including system and mpv version.
- [X] I produced the log file with the exact same set of files, parameters, and conditions used in "Reproduction Steps", with the addition of
--log-file=output.txt. - [X] I produced the log file while the behaviors described in "Actual Behavior" were actively observed.
- [X] I attached the full, untruncated log file.
- [X] I attached the backtrace in the case of a crash.
It's not that the color is incorrect, the subtitles aren't rendered at all. Only the purple drop shadow is rendered. You can get it to render at arbitrary zoom and panscan though, which is odd
edit:
There's a mask(?) to hide this part of the original video, when enough of the mask is off screen, the subtitle is rendered correctly.
It's not that the color is incorrect, the subtitles aren't rendered at all. Only the purple drop shadow is rendered. You can get it to render at arbitrary zoom and panscan though, which is odd
edit:
There's a mask(?) to hide this part of the original video, when enough of the mask is off screen, the subtitle is rendered correctly.
Yeah I shouldn't said frames I meant layers. I assume the GPU rendering is dropping certain layers in the ass file.
when enough of the mask is off screen, the subtitle is rendered correctly.
The problem is that even the mask itself isn't rendered correctly if the video is scaled. In the screenshots provided, the masked text can still be seen faintly, but it should not be visible at all with correct rendering.
Due to the way subtitles are rendered in mpv, only blend-subtitles=video (which is how VSFilter renders subtitles but isn't supported in gpu-next) can render this correctly. See the following screenshot (vo=gpu blend-subtitles=video, video scaled to 1600x900)
Due to the way subtitles are rendered in mpv, only blend-subtitles=video (which is how VSFilter renders subtitles but isn't supported in gpu-next) can render this correctly.
It is supported in a way that it would be rendered relative to the video, but blended after scaling, which is in theory fine, if the scaling is done correctly. That said, it is likely fixable, even without blending at video size. Thoughts?
It is supported in a way that it would be rendered relative to the video, but blended after scaling, which is in theory fine, if the scaling is done correctly. That said, it is likely fixable, even without blending at video size. Thoughts?
This is correct. But the crucial thing is to render all subtitle parts onto one buffer at the subtitle's resolution (which is usually the same as the video resolution for "good" files), and then scale the whole buffer to output size. See the commit message of https://github.com/mpv-player/mpv/commit/dec73f503f948d61e30f868eea4348e26a3ea804 for more details.
Unfortunately, this means that one extra copy is needed for subtitle contents. blend-subtitles=video, as noted above, isn't 100% correct because it fails when video and subtitle resolutions don't match, but it doesn't involve an extra copy.
isn't 100% correct because it fails when video and subtitle resolutions don't match
This is the case where it should render subtitles to proper rect. It doesn't mean we have to blend before, it means that we shouldn't use video rect directly.
It doesn't mean we have to blend before
This is almost true, but due to the boundary condition issue mentioned, scaling the video and the "combined" subtitle plane separately and then blend them later gives different results from blending before scaling.
Additionally, doing it before blending has an advantage that the scale shaders only need to be applied once to the blended image. Currently, it seems that all image subtitles and overlays are always rendered with naive bilinear scaling regardless of the scaler used for video scaling, which is low quality, unless blend-subtitles=video is used. It's possible to achieve almost the same effect by also scaling subtitle plane with shaders, but it's more performance intensive and the boundary condition issue remains.
Not suggesting that blend-subtitles=video have to be implemented for gpu_next, just listing some of the trade offs it needs to make to avoid that.