[SVT-HEVC]Some test found a random issue that the encoded streams are different between BufferedInput enable and disable in encMode[0-3] for P422/P444, and the encoded streams also have run to run issue.

[GuCaixuan]

[Test environment] OS: Ubuntu 18.04.2 LTS SVT CommitID: f8f2a91

During run SVT-HEVC test with Commit ID f8f2a91, found the the encoded streams are different between BufferedInput enable and disable in encMode[0-3]for P422/P444, and the encoded streams also have run to run issue. The issue is a random issue.

encoders/SvtHevcEncApp -i Netflix_Crosswalk_3840x2160_10bit_60Hz_P422.yuv -b buffered_test_M0_Netflix_Crosswalk_3840x2160_10bit_60Hz_P422_Q37_40.265 -w 3840 -h 2160 -bit-depth 10 -fps 60 -intra-period 55 -n 40 -encMode 0 -compressed-ten-bit-format 0 -q 37 -color-format 2 -rc 0 -nb 40 encoders/SvtHevcEncApp -i /home/media/yuvs/SVT/yuvs/Netflix_Crosswalk_3840x2160_10bit_60Hz_P422.yuv -b buffered_test_M0_Netflix_Crosswalk_3840x2160_10bit_60Hz_P422_Q37_-1.265 -w 3840 -h 2160 -bit-depth 10 -fps 60 -intra-period 55 -n 40 -encMode 0 -compressed-ten-bit-format 0 -q 37 -color-format 2 -rc 0 -nb -1

encoders/SvtHevcEncApp -i Netflix_Crosswalk_3840x2160_10bit_60Hz_P444.yuv -b buffered_test_M3_Netflix_Crosswalk_3840x2160_10bit_60Hz_P444_Q40_-1.265 -w 3840 -h 2160 -bit-depth 10 -fps 60 -intra-period 55 -n 40 -encMode 3 -compressed-ten-bit-format 0 -q 40 -color-format 3 -rc 0 -nb -1 encoders/SvtHevcEncApp -i Netflix_Crosswalk_3840x2160_10bit_60Hz_P444.yuv -b buffered_test_M3_Netflix_Crosswalk_3840x2160_10bit_60Hz_P444_Q40_40.265 -w 3840 -h 2160 -bit-depth 10 -fps 60 -intra-period 55 -n 40 -encMode 3 -compressed-ten-bit-format 0 -q 40 -color-format 3 -rc 0 -nb 40

[Austin-Hu]

Hi @GuCaixuan , please help verify that the PR #375 can fix this issue. Thanks!

[liweki]

Hi, Austin, Thanks for your reply.

I try to verify the issue with PR #375 code using commit ID 4225368, found the issue is still reproducible.

Detail test result as below: (test command is same as above)

• For issue: encoded streams are different between BufferedInput enable and disable
• For r2r issue:

Also run the same command with master code b550f13, the issue is also reproducible.

Anything concern, please let me know. Thanks.

[Austin-Hu]

Hi @liweki ,

At first, we should compare checksums (MD5, CRC, etc.) of files, rather than their sizes to check whether they're different or not.

I didn't find the 2 YUV files @GuCaixuan reported, so I used the YUV420 format one (Netflix_Crosswalk_3840x2160_10bit_60Hz_P420.yuv) to debug this issue. Actually I didn't reproduce the issue with "-nb -1" and "-nb 40", and I didn't reproduce the R2R issue, either. But, I indeed reproduced the MD5 mismatch issue with "-nb N" where 1 <= N < 40.

So with the PR #375 applied, the issue I reproduced can be fixed, with the MD5 results below:

where I set "-nb" with -1, 20, 40 and didn't set it separately. And, I didn't reproduce the R2R issue, with or without PR#375 applied.

BTW, I generated a 3840x2160 10bit YUV444 format YUV file via gstreamer to confirm the issue has been fixed.

So, could you help share the 2 YUV files? And could you double check whether the compiled SvtHevcEncApp has been updated or not, as PR #375 fixed the issue in that application, not in the encoder library.

Thanks!

[Austin-Hu]

Hi @GuCaixuan & @liweki ,

After our confirmation, if the "-nb" option is set, the SvtHevcEncApp would repeat encode its specified number of frames. It’s expected that the encoded bitstream with “-nb” option with [1, M) would be different with the one encoded from whole M (M > 1) frames.

So PR #375 will be reverted, and this issue would be marked as "wontfix". Thanks!

[GuCaixuan]

Hi, @Austin-Hu Thank you very mach for your reply.

On our command line, "-n" and "-nb" are same values(-n 40 -nb 40) when BufferedInput is enabled, and -n is also set to 40 when BufferedInput is disabled(-nb -1). The issue will not reproduce for P420 color format, but it will reproduce randomly for P422/P444 color format with encMode[0-3].

Thanks

[Austin-Hu]

Hi @GuCaixuan ,

Because when tracing this issue, I didn’t find the 2 reported YUV422/444 files in your share folder, I just tried the YUV422/444 clips I had and found that there was MD5 mismatch issue with “-nb [1, N)”, with which YUV420 had the same issue. So that I just focused on debugging with YUV420. Sorry for the confusing…

Just now I also reproduced the R2R issue for YUV422/444, no matter “-nb” is set with -1, 40 or not set, under encMode [0, 3]. And I confirmed that it’s a regression issue caused by the PR #294, targeting to address Issue #293.

@kelvinhu325 , please help continue tracing Issue #293 one you have time. Thanks!

[kelvinhu325]

Hi @Austin-Hu , the PR #294 only impact YUV422/444, could provide cmdline in your tests? without PR #294 , YUV422/444 encMode0~3 gets very bad quality, this change is necessary to disable chroma search for YUV422/444 chroma pixels access error, looks like YUV420 and YUV422/444 facing the same issue.

[Austin-Hu]

Hi @kelvinhu325 ,

Please refer to the command lines reported by @GuCaixuan , or you can refer to my command line to reproduce it:

./SvtHevcEncApp -i DucksTakeOff_1280x720_8bit_50Hz_P422.yuv -b 0.265 -w 1280 -h 720 -bit-depth 8 -fps 60 -intra-period 55 -encMode 3 -n 40 -compressed-ten-bit-format 0 -q 37 -color-format 2 -rc 0 -dlf 1 -sao 0 (-nb -1 or 40)

PR #294 is a work around to "fix" the bad quality Issue #293, and it causes this R2R Issue #371. So we need to revert the PR #294 , and ask for your help to continue debugging Issue #293 once you have time. Thanks!

[kelvinhu325]

Hi @kelvinhu325 ,

Please refer to the command lines reported by @GuCaixuan , or you can refer to my command line to reproduce it:

./SvtHevcEncApp -i DucksTakeOff_1280x720_8bit_50Hz_P422.yuv -b 0.265 -w 1280 -h 720 -bit-depth 8 -fps 60 -intra-period 55 -encMode 3 -n 40 -compressed-ten-bit-format 0 -q 37 -color-format 2 -rc 0 -dlf 1 -sao 0 (-nb -1 or 40)

PR #294 is a work around to "fix" the bad quality Issue #293, and it causes this R2R Issue #371. So we need to revert the PR #294 , and ask for your help to continue debugging Issue #293 once you have time. Thanks!

[kelvinhu325]

I reproduced this run2run issue, first difference is at POC35 CU(64, 320) for two tests, the same MV, one get Merge mode winner, another gets Skip, Merge mode has TU data, looks like coefficients data run2run cause this issue.