questdb
questdb copied to clipboard
chore(ilp): optimize parser code on hot path
WIP
Intermediate TSBS benchmarks on local machine.
Before:
puzpuzpuz@apechkurov-laptop:~/projects/tsbs/bin$ GOMAXPROCS=2 ./tsbs_load_questdb --file /tmp/data --workers 1
time,per. metric/s,metric total,overall metric/s,per. row/s,row total,overall row/s
1655103689,2429550.36,2.430000E+07,2429550.36,242955.04,2.430000E+06,242955.04
1655103699,2499935.04,4.930000E+07,2464739.90,249993.50,4.930000E+06,246473.99
1655103709,2440140.53,7.370000E+07,2456541.00,244014.05,7.370000E+06,245654.10
1655103719,2389846.52,9.760000E+07,2439867.22,238984.65,9.760000E+06,243986.72
^C
puzpuzpuz@apechkurov-laptop:~/projects/tsbs/bin$ GOMAXPROCS=2 ./tsbs_load_questdb --file /tmp/data --workers 1
time,per. metric/s,metric total,overall metric/s,per. row/s,row total,overall row/s
1655103798,3679632.82,3.680000E+07,3679632.82,367963.28,3.680000E+06,367963.28
1655103808,3669999.70,7.350000E+07,3674816.50,366999.97,7.350000E+06,367481.65
1655103818,3610246.50,1.096000E+08,3653294.86,361024.65,1.096000E+07,365329.49
1655103828,3739876.95,1.470000E+08,3674940.75,373987.70,1.470000E+07,367494.07
^C
puzpuzpuz@apechkurov-laptop:~/projects/tsbs/bin$ GOMAXPROCS=2 ./tsbs_load_questdb --file /tmp/data --workers 1
time,per. metric/s,metric total,overall metric/s,per. row/s,row total,overall row/s
1655104001,3329710.60,3.330000E+07,3329710.60,332971.06,3.330000E+06,332971.06
1655104011,3180113.59,6.510000E+07,3254916.68,318011.36,6.510000E+06,325491.67
1655104021,2439851.58,8.950000E+07,2983221.93,243985.16,8.950000E+06,298322.19
1655104031,2470115.39,1.142000E+08,2854953.38,247011.54,1.142000E+07,285495.34
1655104041,2449927.56,1.387000E+08,2773947.36,244992.76,1.387000E+07,277394.74
1655104051,2429975.33,1.630000E+08,2716619.11,242997.53,1.630000E+07,271661.91
^C
After:
puzpuzpuz@apechkurov-laptop:~/projects/tsbs/bin$ GOMAXPROCS=2 ./tsbs_load_questdb --file /tmp/data --workers 1
time,per. metric/s,metric total,overall metric/s,per. row/s,row total,overall row/s
1655104218,2289775.51,2.290000E+07,2289775.51,228977.55,2.290000E+06,228977.55
1655104228,2340160.38,4.630000E+07,2314965.85,234016.04,4.630000E+06,231496.58
1655104238,2619890.98,7.250000E+07,2416609.38,261989.10,7.250000E+06,241660.94
1655104248,2680036.03,9.930000E+07,2482464.21,268003.60,9.930000E+06,248246.42
1655104258,2580051.56,1.251000E+08,2501981.14,258005.16,1.251000E+07,250198.11
1655104268,2319948.45,1.483000E+08,2471641.99,231994.85,1.483000E+07,247164.20
^C
puzpuzpuz@apechkurov-laptop:~/projects/tsbs/bin$ GOMAXPROCS=2 ./tsbs_load_questdb --file /tmp/data --workers 1
time,per. metric/s,metric total,overall metric/s,per. row/s,row total,overall row/s
1655104344,2959728.73,2.960000E+07,2959728.73,295972.87,2.960000E+06,295972.87
1655104354,2779998.84,5.740000E+07,2869867.88,277999.88,5.740000E+06,286986.79
1655104364,2790012.71,8.530000E+07,2843250.39,279001.27,8.530000E+06,284325.04
1655104374,3279948.63,1.181000E+08,2952423.84,327994.86,1.181000E+07,295242.38
1655104384,2990232.14,1.480000E+08,2959984.88,299023.21,1.480000E+07,295998.49
1655104394,2779797.71,1.758000E+08,2929951.99,277979.77,1.758000E+07,292995.20
^C
puzpuzpuz@apechkurov-laptop:~/projects/tsbs/bin$ GOMAXPROCS=2 ./tsbs_load_questdb --file /tmp/data --workers 1
time,per. metric/s,metric total,overall metric/s,per. row/s,row total,overall row/s
1655104453,2099837.06,2.100000E+07,2099837.06,209983.71,2.100000E+06,209983.71
1655104463,1990007.35,4.090000E+07,2044924.44,199000.74,4.090000E+06,204492.44
1655104473,2430062.96,6.520000E+07,2173298.56,243006.30,6.520000E+06,217329.86
1655104483,2359869.49,8.880000E+07,2219942.67,235986.95,8.880000E+06,221994.27
1655104493,1969981.45,1.085000E+08,2169951.08,196998.14,1.085000E+07,216995.11
^C
[PR Coverage check]
:heart_eyes: pass : 22 / 23 (95.65%)
file detail
path | covered line | new line | coverage | |
---|---|---|---|---|
:large_blue_circle: | io/questdb/cutlass/line/tcp/LineTcpParser.java | 22 | 23 | 95.65% |
I'll keep this PR pending for some time. The thing is that there is a source of jitter in the scenario under test and I'd like to understand and fix it first. Otherwise, it's very problematic to judge on any optimizations.
Closing this one since it's been handing in the air for a while. Hope to return back to parser optimization one day.