rav1e icon indicating copy to clipboard operation
rav1e copied to clipboard

Have performance information in README

Open rektide opened this issue 6 years ago • 6 comments

In #449 there was an ask for better information about the cpu used to get performance numbers.

In #730, instead of adding that information, all performance information was removed.

Now folks coming by really have absolutely no idea what performance is. Most users searching for performance (like I did) will find an old reddit thread quoting the readme when it said 480@5fps, versus the more recent & now removed 480@10fps, so this removal of all information was extra dangerous.

I would like performance information in the readme.

rektide avatar Nov 28 '18 18:11 rektide

So, what would the testing conditions be? Preferred clip, speed settings, resolution? I can run some encodes tomorrow morning if nobody steps in in the meantime. The CPU is an i7-4770 @ 3.40 GHz

SmilingWolf avatar Nov 28 '18 22:11 SmilingWolf

We could just dump this sort of thing in a wiki or somewhere more appropriate for "moving target" type info.

tmatth avatar Nov 28 '18 22:11 tmatth

It's totally okay to have it in the repo (that way it's version tracked with the code), just someone needs to update it regularly. I don't have any particular preferences on what content is tested - though we may want to think about what default speed we advertise. -s 10 will give us the best fps number, though the quality is poor at that speed.

tdaede avatar Nov 28 '18 22:11 tdaede

We could have something similar to what exists in oxipng, which runs a comparison between oxipng and optipng using hyperfine and updates the readme with the results (including what CPU performed the testing). In this case we could compare with the reference implementation or dav1d if we wanted, or post just rav1e's results. The downside of course is AV1 encoding is much slower than PNG encoding, so running the script could be a little slow.

shssoichiro avatar Nov 29 '18 00:11 shssoichiro

In the meantime: CPU: i7-4770 @ 3.40 GHz rav1e 0.1.0 48e5330 Win64 build from AppVeyor

# rav1e -s 5 -o /dev/null test.y4m
640x480 @ 24000/1001 fps
encoded 312 frames, 6.74 fps, 174.39 Kb/s  
Key Frames:      3    avg size:   12958 B  
Inter:         309    avg size:     814 B  
Intra Only:      0    avg size:       0 B  
Switch:          0    avg size:       0 B

# rav1e -s 4 -o /dev/null test.y4m
640x480 @ 24000/1001 fps
encoded 312 frames, 3.80 fps, 160.58 Kb/s
Key Frames:      3    avg size:   14552 B
Inter:         309    avg size:     724 B
Intra Only:      0    avg size:       0 B
Switch:          0    avg size:       0 B

# rav1e -s 3 -o /dev/null test.y4m
640x480 @ 24000/1001 fps
encoded 312 frames, 2.88 fps, 128.46 Kb/s
Key Frames:      3    avg size:   12078 B
Inter:         309    avg size:     575 B
Intra Only:      0    avg size:       0 B
Switch:          0    avg size:       0 B

SmilingWolf avatar Dec 02 '18 23:12 SmilingWolf

We have started documenting encoding performance in release announcements. See for example: https://github.com/xiph/rav1e/releases/tag/v0.4.0

barrbrain avatar Jan 29 '21 07:01 barrbrain