CompressAI icon indicating copy to clipboard operation
CompressAI copied to clipboard

The very difference result between x265 and the official reference software HM

Open itmanhieu opened this issue 2 years ago • 2 comments

I am wondering the result of benchmark of HM vs x265. When I test (run) your code for x265 for HEVC class B BasketballDrive_1920x1080_P420.yuv sequence with QP=32, number of encoding frame = 100. I got the result with bitrate= 16550. 468, psnr-yuv= 36.589 but when I run the offical HM reference software, the bitrate = 2760.671 (kpps). it with psnr-yuv=36.413. Why your software performance is so low? Is there any bug?

itmanhieu avatar Feb 12 '23 03:02 itmanhieu

Can you provide the commands used for both tests?

Also, 16550.468/2760.671 = 5.995 ≈ 6, so perhaps there is a missing multiplication factor in the measurement somewhere?

YodaEmbedding avatar Feb 12 '23 06:02 YodaEmbedding

for x265, I use the command as in Readme file python3 -m compressai.utils.video.bench x265 -q 32 dataset_path output_dir In the file compressai/utils/video/bench/__main__py I change the variable num_frames = 100 to encode (test) only 100 frame, not the whole sequence. For the official HM (HEVC reference software), I test for both low delay (bitrate = 2760 as mentioned above) and random access (bitrate = 2550) with QP=32, GoP size=16, the active reference picture = 1, all other parameters are default as the configuration file. At first, I have the same thought as you, but all bit rate unit are kbps. that surprises me. Could you test again?

itmanhieu avatar Feb 13 '23 01:02 itmanhieu