FFmpeg icon indicating copy to clipboard operation
FFmpeg copied to clipboard

sao asm mismatch with hevc

Open nuomi2021 opened this issue 2 years ago • 2 comments

There are several code mismatches with hevc. like https://github.com/ffvvc/FFmpeg/blob/main/libavcodec/x86/vvc_sao.asm#L193, the EDGE_SRCSTRIDE and https://github.com/ffvvc/FFmpeg/blob/main/libavcodec/x86/vvc_sao.asm#L173 @zackerthescar do you know why?

thank you

nuomi2021 avatar Aug 13 '23 00:08 nuomi2021

https://github.com/ffvvc/FFmpeg/blob/main/libavcodec/x86/vvc_sao.asm#L193

I had found out that VVC's edge stride was double that of HEVC's, but I didn't realize that this is because of the PB size. Applied a proper fix in #121

https://github.com/ffvvc/FFmpeg/blob/main/libavcodec/x86/vvc_sao.asm#L173

HEVC SAO uses only SSSE3 functions for the 8 and 16 case regardless of if the CPU supports AVX2. Since we're targeting AVX2, we have to declare these functions as AVX2 instead.

Declaring VVC_SAO_EDGE_FILTER 8,0,u after INIT_YMM results in compilation errors, because of the if codeblock on 281-287.

zackerthescar avatar Aug 13 '23 04:08 zackerthescar

Any machine support avx2 will have ssse3. Due to data length reasons. use avx2 may not get benefits. It's not a fault to use ssse3 in this case

nuomi2021 avatar Aug 13 '23 05:08 nuomi2021