Hyperscript.jl icon indicating copy to clipboard operation
Hyperscript.jl copied to clipboard

Add a simple benchmark suite

Open schneiderfelipe opened this issue 3 years ago • 4 comments

This adds a simple benchmark suite (resolves #19). A simple report can be generated as well, with graphs such as

Captura de tela de 2021-05-18 00-33-25

The report can be generated with

julia> include("benchmark/benchmarks.jl")
savereport (generic function with 2 methods)

julia> savereport(run(suite))

Most of the benchmarks are very naive and would benefit from critical feedback.

schneiderfelipe avatar May 18 '21 03:05 schneiderfelipe

Hey,I generally think the Julia project's benchmarking practices are inadequate to correctly evaluate the impact of code changes, in particular the focus on minimum times, and stand by my comment about being surprised if Hyperscript offers performance advantages over any package specifically written to have good performance. :)

I'm also increasingly not the best person to continue to maintain this package, as I haven't used Julia's web ecosystem in a long time. I wonder if there's a possibility of having maintainership taken over by e.g. the folks at JuliaWeb...

yurivish avatar May 18 '21 05:05 yurivish

Hey,I generally think the Julia project's benchmarking practices are inadequate to correctly evaluate the impact of code changes, in particular the focus on minimum times,...

That's OK. I understand your point. How could I change this PR to reflect a better way of evaluating the impact of code changes?

...and stand by my comment about being surprised if Hyperscript offers performance advantages over any package specifically written to have good performance. :)

This could change in the future. #13 is an example. I'm interested in working on improving the performance of Hyperscript.jl, if you don't mind. But, for that, a way of measuring performance is required.

schneiderfelipe avatar May 18 '21 13:05 schneiderfelipe

I'm also increasingly not the best person to continue to maintain this package, as I haven't used Julia's web ecosystem in a long time. I wonder if there's a possibility of having maintainership taken over by e.g. the folks at JuliaWeb...

@yurivish Is there someone we could talk to to make this transfer? I could send a message to #web Slack, if you don't mind.

schneiderfelipe avatar May 18 '21 13:05 schneiderfelipe

@yurivish Is there someone we could talk to to make this transfer? I could send a message to #web Slack, if you don't mind.

Sure, thanks @schneiderfelipe.

How could I change this PR to reflect a better way of evaluating the impact of code changes?

The biggest improvement would be showing distributions instead of individual measurements. E.g. each test could instead be shown as a be a collection of tickmarks

untitled (33)

or a violin plot visualizing the distribution of timing values for each test.

yurivish avatar May 18 '21 16:05 yurivish