EgorBo
Native Bindings is x10 Slower
I have tried to add SimdJsonSharp to my serializer perfomrance test suite. See https://github.com/Alois-xx/SerializerTests
When you compile it and let it run on .NET Core Preview 7
D:\Source\git\SerializerTests\bin\Debug\netcoreapp3.0>SerializerTests.exe -test combined -serializer Utf8JsonSerializer,SimdJsonSharpSerializer,SimdJsonSharpSerializerN
Serializer Objects "Time to serialize in s" "Time to deserialize in s" "Size in bytes" FileVersion Framework ProjectHome DataFormat FormatDetails Supports Versioning
SimdJsonSharpSerializer 1000000 0.1310 0.116 35777803 1.5.0.0 .NET Core 3.0.0-preview7-27912-14 https://github.com/EgorBo/SimdJsonSharp based on https://github.com/lemire/simdjson Text Json No
SimdJsonSharpSerializerN 1000000 0.1117 3.139 35777803 1.7.0.0 .NET Core 3.0.0-preview7-27912-14 https://github.com/EgorBo/SimdJsonSharp based on https://github.com/lemire/simdjson Text Json No
Utf8JsonSerializer 1000000 0.1135 0.330 35777803 1.3.7 .NET Core 3.0.0-preview7-27912-14 https://github.com/neuecc/Utf8Json Text Json Yes
I find the native version needs 3s vs 0.1s for the managed version.
When lookint at the data in PerfView I find most time is spent in paserJson. Why this is so costly I cannot tell but it looks wasteful. Can you take a look why this is so much slower? Am I using the library the wrong way?
@Alois-xx there is also a managed version which doesn't have pinvoke overhead (but has some C# vs C overhead). Anyway simdjsonsharp is good for large files (>200kb)
Yes I know about the managed version. Why should one use the native bindings if they are 10x times slower? Does it make sense to interop into such a slow solution if the interop overhead is so huge? The test with 1 million objects results in a ca. 30 MB Json file. I would have expected to get a much higher speedup compared to a normal serializer like Utf8Json.
@Alois-xx can you share some steps how to test that 30Mb json file, a repro project would be perfect.
Clone https://github.com/Alois-xx/SerializerTests. Compile it and see the command line help. It is a serializer performance testing framework to make it easy to compare different serializers.
D:\Source\git\SerializerTests\bin\Release\netcoreapp3.0>SerializerTests.exe
SerializerTests is a serializer performance testing framework to evaluate and compare different serializers for .NET by Alois Kraus
SerializerTests [-Runs dd] -test [serialize, deserialize, combined, firstCall] [-reftracking] [-maxobj dd]
-N 1,2,10000 Set the number of objects to de/serialize which is repeated -Runs times to get stable results.
-Runs Default is 5. The result is averaged where the first run is excluded from the average
-test xx xx can be serialize, deserialize, combined or firstcall to test a scenario for many different serializers
-reftracking If set a list with many identical references is serialized.
-serializer xxx Execute the test only for a specific serializer with the name xxx where multiple ones can be used with ,
-list List all registered serializers
-notouch Do not touch the deserialized objects to test lazy deserialization
To execute deserialize you must first have called the serialize to generate serialized test data on disk to be read during deserialize
Examples
Compare protobuf against MessagePackSharp for serialize and deserialize performance
SerializerTests -Runs 1 -test combined -serializer protobuf,MessagePackSharp
Test how serializers perform when reference tracking is enabled. Currently that are BinaryFormatter,Protobuf_net and DataContract
Although Json.NET claim to have but it is not completely working.
SerializerTests -Runs 1 -test combined -reftracking
Test SimdJsonSharpSerializer serializer with 3 million objects for serialize and deserialize.
SerializerTests -test combined -N 3000000 -serializer SimdJsonSharpSerializer
I have added SimdJsonSharpSerializer with 3 million objects which creates a 100 MB Json file with Utf8JsonSerializer and deserialized it with your parsers.
For code see https://github.com/Alois-xx/SerializerTests/blob/master/Serializers/SimdJsonSharpSerializer.cs and https://github.com/Alois-xx/SerializerTests/blob/master/Serializers/SimdJsonSharpSerializerN.cs
Hi,
I have cloned the solution today and everything did work. Did you build it with a VS 2019 Preview? You need to update VS2019 to 19.2.0 at least to compile
.NET Core 3 libraries with the correct MS Build version which ships with VS2019.
I have the binary also in the same folder
"D:\Source\git\SerializerTests\bin\Release\netcoreapp3.0\runtimes\win-x64\native\SimdJsonNative.dll"
But a dependencies output shows the issue:
This library links to the debug runtime of Visual Studio. If you do not have VS installed on your machine it will not run.
Yours,
Alois
From: Gabriel Bider [email protected] Sent: Sunday, October 20, 2019 1:23 PM To: EgorBo/SimdJsonSharp [email protected] Cc: Alois-xx [email protected]; Mention [email protected] Subject: Re: [EgorBo/SimdJsonSharp] Native Bindings is x10 Slower (#13)
@Alois-xx https://github.com/Alois-xx When running your example, I'm getting an exception DllNotFoundException: Unable to load DLL 'SimdJsonNative' or one of its dependencies: The specified module could not be found. (0x8007007E) on the native tests, what am I missing?
The DLL file is located at SerializerTests\bin\Release\netcoreapp3.0\runtimes\win-x64\native\SimdJsonNative.dll
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EgorBo/SimdJsonSharp/issues/13?email_source=notifications&email_token=AADDZ5P7FIZV7TF5CMO2SMTQPQ5P7A5CNFSM4IHMUQG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBYH4WA#issuecomment-544243288 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AADDZ5NC6T45FD5AMCY5VP3QPQ5P7ANCNFSM4IHMUQGQ . https://github.com/notifications/beacon/AADDZ5J56JZZGWXS2BCYKHTQPQ5P7A5CNFSM4IHMUQG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBYH4WA.gif
PS: This is the debug runtime of Visual Studio 2015. If you want drill deeper you should recompile the binary anyways to get the pdbs.
From: Alois Kraus [email protected] Sent: Monday, October 21, 2019 9:45 PM To: 'EgorBo/SimdJsonSharp' [email protected]; 'EgorBo/SimdJsonSharp' [email protected] Cc: 'Mention' [email protected] Subject: RE: [EgorBo/SimdJsonSharp] Native Bindings is x10 Slower (#13)
Hi,
I have cloned the solution today and everything did work. Did you build it with a VS 2019 Preview? You need to update VS2019 to 19.2.0 at least to compile
.NET Core 3 libraries with the correct MS Build version which ships with VS2019.
I have the binary also in the same folder
"D:\Source\git\SerializerTests\bin\Release\netcoreapp3.0\runtimes\win-x64\native\SimdJsonNative.dll"
But a dependencies output shows the issue:
This library links to the debug runtime of Visual Studio. If you do not have VS installed on your machine it will not run.
Yours,
Alois
From: Gabriel Bider <[email protected] mailto:[email protected] > Sent: Sunday, October 20, 2019 1:23 PM To: EgorBo/SimdJsonSharp <[email protected] mailto:[email protected] > Cc: Alois-xx <[email protected] mailto:[email protected] >; Mention <[email protected] mailto:[email protected] > Subject: Re: [EgorBo/SimdJsonSharp] Native Bindings is x10 Slower (#13)
@Alois-xx https://github.com/Alois-xx When running your example, I'm getting an exception DllNotFoundException: Unable to load DLL 'SimdJsonNative' or one of its dependencies: The specified module could not be found. (0x8007007E) on the native tests, what am I missing?
The DLL file is located at SerializerTests\bin\Release\netcoreapp3.0\runtimes\win-x64\native\SimdJsonNative.dll
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EgorBo/SimdJsonSharp/issues/13?email_source=notifications&email_token=AADDZ5P7FIZV7TF5CMO2SMTQPQ5P7A5CNFSM4IHMUQG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBYH4WA#issuecomment-544243288 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AADDZ5NC6T45FD5AMCY5VP3QPQ5P7ANCNFSM4IHMUQGQ . https://github.com/notifications/beacon/AADDZ5J56JZZGWXS2BCYKHTQPQ5P7A5CNFSM4IHMUQG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBYH4WA.gif