More recent data

[JustinDrake]

Would it be possible to redraw the charts with more recent data?

[damageboy]

I've just done so here at this fork (updated the source code to support spec2017, hopefully didn't mess up): https://github.com/damageboy/analyze-spec-benchmarks

You can pretty much run the scripts according to the instructions and get everything working.

If you just want updated charts, I've just added them to my commit at my fork:

https://github.com/damageboy/analyze-spec-benchmarks#charts-generated-on-2019-05-21

[HenkPoley]

@damageboy something makes me think that the 2017 FP results are stretched, and needs to be scaled smaller (and then offset upwards).

Or at least the spread is much higher than before, which is of course nice for a benchmark number.

[damageboy]

@HenkPoley I'm not sure I'm following...

Do you mean that you think there's an issue with how it is displayed? Or an issue with how the 2017 results are normalized to the previous generations?

[HenkPoley]

The spread for the the 2017 floating point data is much higher. I'm not sure, but it feels to me I would try to correct for it. But on the other hand, it's just the data. They probably added tests specifically to enlarge differences, or maybe changed the score accumulation algorithm (I know that the entirely different benchmark Speedometer 2.0 changed theirs, for example).

Edit: Now it seems that in 2017 things suddenly improved very much. Due to difference in the scoring. That's what I'm talking about

[HenkPoley]

Maybe switch the graph background, similar to the earlier boundary.

[HenkPoley]

Ported @damageboy version to Python 3.8, and added some workaround for recent data:

https://github.com/HenkPoley/analyze-spec-benchmarks/tree/37f3c74d0422bae21a71fca892be5bd27998467b