[FATAL ERROR] VMAF own flaws

[antatura]

My poor English, forgive me.

I like your project! it's easy to compare videos.

Here, I think VMAF is not fully developed yet.

Two same SHA256 1080p video's VMAF result is 99......., not 100

However, Two same SHA256 4K video's VMAF result is 100

And, The frame's pts_time also affect VMAF result.

FFmpeg -i whatever.mp4 -s 1920x1080 -r 7 -t 5 refs.mp4

FFmpeg -i refs.mp4 -c copy refs-copy.mkv

FFmpeg -i refs.mp4 -c:v ffvhuff lossless.mkv

ffvhuff is a lossless encoder.

FFmetries compare refs-copy.mkv to refs.mp4, VMAF=83......

FFmetries compare lossless.mkv to refs.mp4, VMAF=83......

Oh! it shouldn't be. There're almost one video.

But their frame's pts_time are different.

ffprobe -v 16 -show_entries frame=pts_time -of csv=p=0 [input]

VMAF request pts_time same totally......

Here is my solution:

FFmpeg -r 1 -i lossless.mkv -r 1 -i Refs.mp4 -map v -lavfi libvmaf -f null -

VMAF Result: 99.........

Hope this helps you！

Thanks!

[fifonik]

Thanks for feedback. I will play with the -r option you suggested.

[henanzheng]

Thank @ fifonik for providing this good tool. Thank you for your efforts! I also encountered the problem that the test value of MKV container is low. Recently, I encountered the same problem with qsv-av1. The tool used is handbrake, the encoder is intel-qsv-av1, the package is WebM, and the Intel arc A380 graphics card. You can also check it together.

[henanzheng]

Handbrake development Edition

[fifonik]

In version 1.4.5b -r <framerate> is added to ffmpeg arguments + it is possible to fine-tune all arguments in FFMetrics.conf (see FFMetrics.conf.example)

It would be nice if you can check it out.

Thanks.