FSharp.Json
FSharp.Json copied to clipboard
fsprojects
[Don't Merge] Benchmarks
I've adopted Benchmark project from FsPickler. There are some benchmarks that you may find interesting. Some benchmarks fail because of lack of support. Here are results:
FSharp.Json.Benchmarks.Array3D.Array3DRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 7.737 ms | 0.1019 ms | 0.0903 ms |
| FSharp.Json | NA | NA | NA |
Benchmarks with issues: Array3DRoundtrip.FSharp.Json: DefaultJob
FSharp.Json.Benchmarks.BoxedArray.BoxedArrayRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 10.73 us | 0.2112 us | 0.3096 us |
| FSharp.Json | 31.65 us | 0.6303 us | 0.9433 us |
FSharp.Json.Benchmarks.Dictionary.DictionaryRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 6.951 ms | 0.0966 ms | 0.0904 ms |
| FSharp.Json | NA | NA | NA |
Benchmarks with issues: DictionaryRoundtrip.FSharp.Json: DefaultJob
FSharp.Json.Benchmarks.ExceptionBench.ExceptionRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 182.3 us | 3.463 us | 3.402 us |
| FSharp.Json | NA | NA | NA |
Benchmarks with issues: ExceptionRoundtrip.FSharp.Json: DefaultJob
FSharp.Json.Benchmarks.FloatArray.FloatArrayRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 79.14 us | 1.580 us | 1.622 us |
| FSharp.Json | NA | NA | NA |
Benchmarks with issues: FloatArrayRoundtrip.FSharp.Json: DefaultJob
FSharp.Json.Benchmarks.FSharpBinTree.FSharpBinaryRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 43.78 ms | 0.8672 ms | 1.271 ms |
| FSharp.Json | 329.74 ms | 6.4648 ms | 8.176 ms |
FSharp.Json.Benchmarks.FSharpList.FSharpListRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 255.5 us | 4.816 us | 4.946 us |
| FSharp.Json | 1,378.4 us | 27.255 us | 33.471 us |
FSharp.Json.Benchmarks.FSharpMap.FSharpMapRountrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 46.07 us | 0.8742 us | 1.007 us |
| FSharp.Json | 226.30 us | 4.4776 us | 5.822 us |
FSharp.Json.Benchmarks.FSharpSet.FSharpSetRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 23.95 us | 0.3331 us | 0.2953 us |
| FSharp.Json | NA | NA | NA |
Benchmarks with issues: FSharpSetRoundtrip.FSharp.Json: DefaultJob
FSharp.Json.Benchmarks.ISerializable.ISerializableRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 5.921 us | 0.1181 us | 0.2567 us |
| FSharp.Json | NA | NA | NA |
Benchmarks with issues: ISerializableRoundtrip.FSharp.Json: DefaultJob
FSharp.Json.Benchmarks.LargeObject.LargeFSharpValueRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 449.0 us | 1.109 us | 1.037 us |
| FSharp.Json | NA | NA | NA |
Benchmarks with issues: LargeFSharpValueRoundtrip.FSharp.Json: DefaultJob
FSharp.Json.Benchmarks.LargeTuple.LargeTupleRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 22.29 us | 0.0941 us | 0.0880 us |
| FSharp.Json | 68.44 us | 0.1671 us | 0.1563 us |
FSharp.Json.Benchmarks.MemberInfo.MemberInfoRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
| Method | Mean | Error |
|---|---|---|
| Newtonsoft.Json | NA | NA |
| FSharp.Json | NA | NA |
Benchmarks with issues: MemberInfoRoundtrip.Newtonsoft.Json: DefaultJob MemberInfoRoundtrip.FSharp.Json: DefaultJob
FSharp.Json.Benchmarks.Poco.PocoRoundtrip
BenchmarkDotNet=v0.11.5, OS=ubuntu 16.04
Intel Core i5-7400 CPU 3.00GHz (Kaby Lake), 1 CPU, 4 logical and 4 physical cores
.NET Core SDK=2.2.203
[Host] : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT DEBUG
DefaultJob : .NET Core 2.2.4 (CoreCLR 4.6.27521.02, CoreFX 4.6.27521.01), 64bit RyuJIT
| Method | Mean | Error | StdDev |
|---|---|---|---|
| Newtonsoft.Json | 9.588 us | 0.0425 us | 0.0377 us |
| FSharp.Json | NA | NA | NA |
Benchmarks with issues: PocoRoundtrip.FSharp.Json: DefaultJob
Attached run log so you could see exceptions. Waiting for your suggestions on benchmarks. Maybe add some other libraries to compare? BenchmarkRun-20190429-105157.log
Thank you for doing this!
First thing to note: performance has never been priority for me while writing FSharp.Json. This does not mean I made it slow intentionally, I just never concentrated on it. Main focus was on strong type consistency: correct null-safety support, better unions support.
Main reason of slowness (you are right) is in the slow backend which is JsonValue. JsonValue looked as convenient intermediate layer between raw data and F# structures: string (or byte array) -> JsonValue -> F# structures. JsonValue provided me some luxury in FSharp.Json source code, though this convenience came with performance penalty. Newtonsoft.Json is using simplest tokenizer under the hood and that's why it's faster.
Switching backend is plenty of work and I think I will need to do it at some point. Especially for leveraging new Span type. Though FSharp.Json wasn't designed with swappable backend in mind and as I mentioned JsonValue is bigger then just tokenizer. So this is not easy. Also regarding backends made in C# (like Utf8Json): I strongly believe in zero transitive dependencies approach for such core libraries as JSON serialization. Internalizing C# code into F# project is not possible as I know. There are some workarounds but they are not as beautiful as internalized code to my taste.
On Mon, Apr 29, 2019, 04:18 Daniil Barbashov [email protected] wrote:
Attached run log so you could see exceptions. Waiting for your suggestions on benchmarks. Maybe add some other libraries to compare? BenchmarkRun-20190429-105157.log https://github.com/vsapronov/FSharp.Json/files/3126473/BenchmarkRun-20190429-105157.log
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/vsapronov/FSharp.Json/pull/23#issuecomment-487488385, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGYLNR4KPDJ5PWBACAAYPTPS2VM3ANCNFSM4HJBGP6Q .
