JSO never performs VSFilter colour mangling

[ThaUnknown]

trafficstars

I remember there being talk about some cases where JSSO has color inaccuracies in the bitmap decode calculation and I reproduced this on [SakuraCircle] Acchi Kocchi - 09 (BD 1920x1080 HEVC FLAC) [CB0433CA].mkv where fd91a0 is calculated as f484a1 @ 15:51

[ThaUnknown]

this is because the .ass subtitles included YCbCr Matrix: TV.601 in their script info. does libass compensate the matrix? or does it not? if it does is there a way to turn that off?

[ThaUnknown]

full file: 4Tsumiki.ass.txt

[TheOneric]

JSO needs to know the video colourspace to accurately deal with most ASS colours. See libass’ docs for ASS_YCbCrMatrix from ass_types.h, somewhere around here: https://github.com/libass/libass/blob/master/libass/ass_types.h#L155

this is because the .ass subtitles included YCbCr Matrix: TV.601 in their script info.

Not putting any YCbCr header and putting an explicit YCbCr Matrix: TV.601 are identical in all compatible ASS renderers.

[ThaUnknown]

JSO needs to know the video colourspace to accurately deal with most ASS colours. See the docs here: https://github.com/libass/libass/blob/b890df23eb8931728acdf240fd5d29e53401b4f2/libass/ass_types.h#L152-L217

this is because the .ass subtitles included YCbCr Matrix: TV.601 in their script info.

Not putting any YCbCr header and putting an explicit YCbCr Matrix: TV.601 are identical in all compatible ASS renderers.

so then is this not the issue causing this? or more like what would be the underlying issue causing this? see: JSSO:

mpv:

[TheOneric]

so then is this not the issue causing this?

It most likely is the cause.

[ThaUnknown]

so then is this not the issue causing this?

It most likely is the cause.

hmmmmm if TV.601 is the default value then I don't really know what else to do here... JSSO simply cant handle this case?

I don't really understand the underlying issue, but I assume normally subs are meant to be layered on top of the video and then color converted by whatever the video color range's are to what's meant to be displayed but JSSO goes around that as it renders on top of a video instead of together with it, which means it never ends up getting that color correction and is where the issue lies?

[ThaUnknown]

@TheOneric is the only way to fix this problem is to know the video metadata, aka color space? I've been trying to read up on this, but haven't gotten much information as to implementation details. I've found a way to extract colorspace metadata from videos in browser, if that is going to be of any help?

[TheOneric]

is the only way to fix this problem is to know the video metadata, aka color space?

Yes, this is a necessary part of the solution.

I've been trying to read up on this, but haven't gotten much information as to implementation details.

The link to the related libass docs in my first reply here already contains the full details. (As there were some rewordings since, I also just updated the link to point at the newest version.)

[ThaUnknown]

is the only way to fix this problem is to know the video metadata, aka color space?

Yes, this is a necessary part of the solution.

I've been trying to read up on this, but haven't gotten much information as to implementation details.

The link to the related libass docs in my first reply here already contains the full details. (As there were some rewordings since, I also just updated the link to point at the newest version.)

I could probably fix this, but I'm not certain what actually would need to be done in this case, could simply changing what color space is given to libass itself in the subtitle file be a potential fix? or does it need to be done after libass spits out its images

edit: ah I see, seems like manual post-processing/color correction is needed for this, yikes

[ThaUnknown]

seems like we could "borrow" some of the MPV code for this https://github.dev/mpv-player/mpv/blob/59fc8eecbc08d22368451b31fcf799aec557ab0d/sub/sd_ass.c

[Yay295]

Looks like it's this function specifically:

https://github.com/mpv-player/mpv/blob/59fc8eecbc08d22368451b31fcf799aec557ab0d/sub/sd_ass.c#L853-L958

[ThaUnknown]

yeah, but there's a lot of other black magic happening there that I honestly can't wrap my head around and I'm not sure what a lot of it does :/

[Yay295]

More stuff here too: https://github.com/mpv-player/mpv/blob/f02ab2aabb630d6618dd99f63af8014c10d79562/video/csputils.c

[ThaUnknown]

@TheOneric if you ever want to come back to this, I can see 2 ways of doing this, the same way that MPV does it, so pretty much steal their code for this, or use SVG filters, TLDR a canvas context can have a filter which can be an URL, which can be an SVG filter, those can do color matrix conversions but I can't find any reliable info on how to do the math for the RGB conversion for this, but the matrixes can do:

     R G B A W
R' | 1 0 0 0 0 |
G' | 0 1 0 0 0 |
B' | 0 0 1 0 0 |
A' | 0 0 0 1 0 |

where W is hard offset

the video colorspace can be extracted via VideoFrame object, which is supported everywhere except firefox via new VideoFrame(htmlVideoElement).colorSpace.matrix

I fixed this in my renderer like this: https://github.com/ThaUnknown/jassub/commit/5047e1811e5c72e70ac17c08635d1b92beba5a53