Pidgin
Pidgin copied to clipboard
Published
20 hours ago •
benjamin-hodgson
benjamin-hodgson
Add Farkle benchmarks.
Hello, I have created a parsing library called Farkle. Its API bears a strong resemblance to parser combinators, but in contrast with ordinary parser combinator libraries like Pidgin, Sprache and FParsec, it uses bottom-up LALR(1) parsing under the hood. It would be fun to add it to the benchmarks, and would help tuning the performance of both Farkle and Pidgin.
I ran it on my machine, and the enclosed results are interesting. In the JSON benchmarks, Farkle is a solid contender, finishing always in the top three in both time and memory, and finishing first in the deep JSON files benchmark. In the expression benchmarks, Farkle always finishes second in time, but uses significantly more memory than Pidgin and FParsec.
Let me know if you have any questions.
JSON benchmarks.
BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19044
Intel Core i7-7700HQ CPU 2.80GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=7.0.100-preview.2.22153.17
[Host] : .NET Core 5.0.15 (CoreCLR 5.0.1522.11506, CoreFX 5.0.1522.11506), X64 RyuJIT
DefaultJob : .NET Core 5.0.15 (CoreCLR 5.0.1522.11506, CoreFX 5.0.1522.11506), X64 RyuJIT
| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|---|
| BigJson_Pidgin | 456.5 us | 5.86 us | 5.20 us | 456.0 us | 1.00 | 0.00 | 33.2031 | - | - | 101.7 KB |
| BigJson_Sprache | 3,579.2 us | 71.09 us | 157.54 us | 3,575.0 us | 7.81 | 0.31 | 1746.0938 | - | - | 5349.63 KB |
| BigJson_Superpower | 2,277.5 us | 37.81 us | 33.52 us | 2,284.5 us | 4.99 | 0.10 | 296.8750 | - | - | 913.43 KB |
| BigJson_FParsec | 361.2 us | 7.22 us | 15.70 us | 359.1 us | 0.81 | 0.04 | 76.6602 | - | - | 235.93 KB |
| BigJson_Farkle | 545.6 us | 7.40 us | 6.92 us | 546.7 us | 1.20 | 0.02 | 68.3594 | - | - | 209.54 KB |
| LongJson_Pidgin | 414.5 us | 8.13 us | 11.40 us | 412.9 us | 1.00 | 0.00 | 33.6914 | - | - | 104.25 KB |
| LongJson_Sprache | 2,783.8 us | 55.17 us | 95.17 us | 2,761.2 us | 6.76 | 0.30 | 1406.2500 | - | - | 4311.36 KB |
| LongJson_Superpower | 1,809.0 us | 13.58 us | 12.04 us | 1,806.1 us | 4.34 | 0.10 | 230.4688 | - | - | 706.79 KB |
| LongJson_FParsec | 314.0 us | 5.43 us | 8.13 us | 311.2 us | 0.76 | 0.03 | 74.7070 | - | - | 230.3 KB |
| LongJson_Farkle | 449.6 us | 4.60 us | 4.08 us | 448.6 us | 1.08 | 0.03 | 57.1289 | - | - | 175.31 KB |
| DeepJson_Pidgin | 533.0 us | 9.77 us | 26.92 us | 524.8 us | 1.00 | 0.00 | 66.4063 | - | - | 205.29 KB |
| DeepJson_Sprache | 3,108.7 us | 63.99 us | 183.59 us | 3,074.9 us | 5.86 | 0.47 | 730.4688 | 222.6563 | - | 2946.56 KB |
| DeepJson_FParsec | 328.3 us | 6.54 us | 17.23 us | 325.7 us | 0.62 | 0.04 | 62.0117 | - | - | 190.43 KB |
| DeepJson_Farkle | 318.7 us | 5.73 us | 8.92 us | 315.9 us | 0.57 | 0.03 | 48.3398 | - | - | 148.17 KB |
| WideJson_Pidgin | 262.9 us | 4.64 us | 4.34 us | 261.6 us | 1.00 | 0.00 | 15.6250 | - | - | 48.42 KB |
| WideJson_Sprache | 1,580.5 us | 25.60 us | 34.17 us | 1,572.4 us | 6.05 | 0.17 | 912.1094 | - | - | 2797.28 KB |
| WideJson_Superpower | 1,145.3 us | 22.73 us | 31.86 us | 1,139.4 us | 4.38 | 0.14 | 148.4375 | - | - | 459.74 KB |
| WideJson_FParsec | 213.2 us | 4.01 us | 4.29 us | 211.8 us | 0.81 | 0.02 | 33.9355 | - | - | 104.63 KB |
| WideJson_Farkle | 350.5 us | 6.99 us | 18.29 us | 345.2 us | 1.32 | 0.06 | 34.1797 | - | - | 105.56 KB |
Expression benchmarks.
BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19044
Intel Core i7-7700HQ CPU 2.80GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=7.0.100-preview.2.22153.17
[Host] : .NET Core 5.0.15 (CoreCLR 5.0.1522.11506, CoreFX 5.0.1522.11506), X64 RyuJIT
DefaultJob : .NET Core 5.0.15 (CoreCLR 5.0.1522.11506, CoreFX 5.0.1522.11506), X64 RyuJIT
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| LongInfixL_Pidgin | 426,214.4 ns | 8,470.92 ns | 12,678.88 ns | 11.78 | 0.34 | - | - | - | 136 B |
| LongInfixR_Pidgin | 422,279.7 ns | 1,854.75 ns | 1,644.19 ns | 11.50 | 0.05 | - | - | - | 128 B |
| LongInfixL_FParsec | 36,710.6 ns | 107.03 ns | 89.37 ns | 1.00 | 0.00 | 0.0610 | - | - | 200 B |
| LongInfixR_FParsec | 42,116.8 ns | 227.04 ns | 212.38 ns | 1.15 | 0.01 | 0.0610 | - | - | 200 B |
| LongInfixL_Farkle | 309,182.5 ns | 908.00 ns | 804.92 ns | 8.42 | 0.04 | 51.2695 | - | - | 160976 B |
| LongInfixR_Farkle | 320,691.2 ns | 1,242.24 ns | 1,161.99 ns | 8.73 | 0.03 | 66.4063 | - | - | 208832 B |
| ShortInfixL_Pidgin | 1,420.0 ns | 5.79 ns | 5.13 ns | 12.88 | 0.08 | 0.0420 | - | - | 136 B |
| ShortInfixR_Pidgin | 1,427.4 ns | 5.90 ns | 5.52 ns | 12.94 | 0.09 | 0.0401 | - | - | 128 B |
| ShortInfixL_FParsec | 110.3 ns | 0.54 ns | 0.48 ns | 1.00 | 0.00 | 0.0637 | - | - | 200 B |
| ShortInfixR_FParsec | 109.9 ns | 0.17 ns | 0.15 ns | 1.00 | 0.00 | 0.0637 | - | - | 200 B |
| ShortInfixL_Farkle | 679.9 ns | 1.70 ns | 1.59 ns | 6.17 | 0.03 | 0.4129 | - | - | 1296 B |
| ShortInfixR_Farkle | 679.8 ns | 2.38 ns | 2.11 ns | 6.17 | 0.04 | 0.4129 | - | - | 1296 B |
