imdbench icon indicating copy to clipboard operation
imdbench copied to clipboard

Add dotnet benchmarks

Open quinchs opened this issue 3 years ago • 7 comments

Adds both edgedb_dotnet and efcore (most popular .NET ORM) as a benchmark target.

Requires .NET 7 SDK to run

quinchs avatar Dec 23 '22 21:12 quinchs

@elprans can you run the benchmarks on the same machine as the others and commit the results?

quinchs avatar Dec 28 '22 16:12 quinchs

Hm. I'm actually getting an error:

Exception in edgedb_dotnet.get_movie: InvalidValueError: invalid input to mean(): not enough elements in input set

elprans avatar Dec 28 '22 18:12 elprans

Exception in edgedb_dotnet.get_movie: InvalidValueError: invalid input to mean(): not enough elements in input set

That error format looks like it was from an ErrorResponse message meaning it was thrown in edgedb itself.

Pulling the movie id it was trying to select and running the query in the CLI throws the same error on my end:

edgedb> SELECT Movie {
.......     id,
.......     image,
.......     title,
.......     year,
.......     description,
.......     avg_rating,
.......     directors: {
.......         id,
.......         full_name,
.......         image,
.......     }
.......     ORDER BY Movie.directors@list_order EMPTY LAST
.......         THEN Movie.directors.last_name,
.......     cast: {
.......         id,
.......         full_name,
.......         image,
.......     }
.......     ORDER BY Movie.cast@list_order EMPTY LAST
.......         THEN Movie.cast.last_name,
.......     reviews := (
.......         SELECT Movie.<movie[IS Review] {
.......             id,
.......             body,
.......             rating,
.......             author: {
.......                 id,
.......                 name,
.......                 image,
.......             }
.......         }
.......         ORDER BY .creation_time DESC
.......     ),
....... }
....... FILTER .id = <uuid>$id;
Parameter <uuid>$id: 5841f480-862b-11ed-be8c-83ddd150e3f2
edgedb error: InvalidValueError: invalid input to mean(): not enough elements in input set

Is this a bug with EdgeDB?

quinchs avatar Dec 28 '22 19:12 quinchs

I've pushed a change to log the ids that are used when an error is thrown

quinchs avatar Dec 28 '22 19:12 quinchs

This seems to be an issue with the data set. There should be at least one review for each movie.

elprans avatar Dec 28 '22 19:12 elprans

Is there a quick fix?

quinchs avatar Dec 28 '22 19:12 quinchs

I looked at this again, and the only diff I can see from other benchmarks is that you are not selecting the ids randomly and instead just take the first slice, which somehow contains problematic data. Randomize id selection to see if we can get a successful run.

elprans avatar May 19 '23 22:05 elprans