How can users distinguish between other benchmark properties than the common ones?

[lauxjpn]

I would like users to be able to differentiate between two different data access libraries, both accessing the same database.

In this concrete case, using MySqlConnector (community developed) will be 5 times as fast as using Connector/NET (Oracle developed) when querying a MySQL database. So I want to make sure, that users are aware of the library that was used to reach the performance of the benchmark (where necessary).

So far, tags and notes look promising, but they don't show up in the visualization. We could just add the library name to the display name of the test for our benchmark cases, but we would need to significantly shorten the library names, to stay within the established naming conventions. And I don't think, users would necessarily know, what mc or cn would mean.

To solve this, it might be enough to show the notes below to the name etc. when hovering over a benchmark name in the results on the website. We could then use an abbreviated name for the library (only where necessary) in the display name, and mention it with its full name in the notes. Or the tags would automatically show below the display name when hovering over it.

/cc @bgrainger

[bgrainger]

Do we really need to maintain .NET benchmarks for MySQL Connector/NET when we know it's slow (and buggy)?

Maybe it'd just be easier to remove those Connector/NET benchmarks, which would have the beneficial side-effect of making the whole test suite a little smaller and execute faster?

[lauxjpn]

Currently, there are no Connector/NET benchmarks at all. (I have only added some in my local repo for comparison purposes.) Independent of, whether Connector/NET benchmarks should be added as well, users should be made aware, that a certain benchmark achieved the published performance using a specific library.

I think this is important information in the MySQL case for users, where there is a 400% difference between Connector/NET and MySqlConnector in under load.

[bgrainger]

Ah, I wasn't paying attention when aspnet-mono-ngx (using MySql.Data 8.0.17) was removed: https://github.com/TechEmpower/FrameworkBenchmarks/commit/fda6b2382c2780b6c6f53699afce27a042564a50

[lauxjpn]

I wasn't even aware, that there ever were Connector/NET tests :)