arrow icon indicating copy to clipboard operation
arrow copied to clipboard

Publish some benchmarks

Open spacemanspiff2007 opened this issue 4 years ago • 5 comments

When searching for different python time libraries there still countless articles popping up who suggest that especially pendulum has a huge speed advantage over arrow. While these might have been true, it's absolutely unclear how the situation has changed since then. It would be nice if there was a small benchmark section in the docs which would compare basic operations.

spacemanspiff2007 avatar Nov 01 '21 04:11 spacemanspiff2007

@jadchaar, I think we should do this.

anishnya avatar Nov 02 '21 00:11 anishnya

Agreed especially so we can compare before and after the Cython work we are planning. Could be cool to constantly update a wiki page (with benchmark graphs) with a CI workflow on every tag and every python version.

jadchaar avatar Nov 02 '21 03:11 jadchaar

Using code from https://aboutsimon.com/blog/2016/08/04/datetime-vs-Arrow-vs-Pendulum-vs-Delorean-vs-udatetime.html and modifying it to run on Python 3.9 I was able to get the following results.

Note - I couldn't get udatetime to install and quite a bit has changed with pendulum making some of the comparisons difficult.

============ benchmark_parse
_datetime 9.640075228000114 [9.763600978999875, 9.668748288000188, 9.640075228000114, 9.760113094999724, 9.717294457999742]
_arrow 67.04907164500037 [67.04907164500037, 68.1874751600003, 67.57100422900021, 67.84225658600008, 67.89600008499974]
_pendulum 6.8031756150003275 [6.8477682889997595, 6.81757830000015, 6.8031756150003275, 6.8112750700001925, 6.854873013000088]
_delorean 79.36898372299993 [79.36898372299993, 79.53761111499989, 79.84778318200006, 80.14455868899995, 80.14512624700001]

============ benchmark_format
_datetime 3.239017418000003 [3.3066458970001804, 3.253738739000255, 3.239017418000003, 3.2690720719997444, 3.2413774930000727]
_arrow 1.4535876850000022 [1.455979157999991, 1.4539373530001285, 1.4550910959997054, 1.4535876850000022, 1.4555275489997257]
_pendulum 1.5255801739999697 [1.5255801739999697, 1.529671358000087, 1.5531356820001747, 1.526431623999997, 1.5340547209998476]
_delorean 3.875935237000249 [3.9699474889998783, 3.992204258000129, 3.959789434999948, 3.875935237000249, 3.8760139239998352]

============ benchmark_utcnow
_datetime 1.2329302449998067 [1.2587823889998617, 1.234189470000274, 1.2447910880000563, 1.2346941650002918, 1.2329302449998067]
_arrow 2.3831869979999283 [2.3915442749998874, 2.38454174699973, 2.3831869979999283, 2.410346625999864, 2.3863246780001646]
_pendulum 2.416928165000172 [2.418707472999813, 2.42483995099974, 2.416928165000172, 2.430766498000139, 2.4310383629999706]
_delorean 2.5732629419999284 [2.574094752000292, 2.574406983999779, 2.579042515999845, 2.57815893399993, 2.5732629419999284]

============ benchmark_now
_datetime 2.1431128250001166 [2.1431128250001166, 2.1466130099997827, 2.1494242970002233, 2.1619246580003164, 2.1500752520000788]
_arrow 32.88162481800009 [32.96679717000006, 32.95545565300017, 32.88708177999979, 32.88162481800009, 32.98677041999963]
_pendulum 3.2865496659997007 [3.33860700599962, 3.301104772999679, 3.291630373000771, 3.2939674620001824, 3.2865496659997007]
/home/chris/.pyenv/versions/bench_arrow/lib/python3.9/site-packages/delorean/dates.py:170: PytzUsageWarning: The normalize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  dt = tz.normalize(dt)
_delorean 6.749951231000523 [6.775615546000154, 6.749951231000523, 6.754888565999863, 6.758163626000169, 6.768949623000481]

============ benchmark_fromtimestamp
_datetime 2.2006504309993034 [2.204797606000284, 2.2577747280010954, 2.2579442980004387, 2.2277943460012466, 2.2006504309993034]
_arrow 32.60892093700022 [33.383653486000185, 33.16171884899995, 32.60892093700022, 33.319603480000296, 33.178806079000424]
_pendulum 3.149250230000689 [3.2103926239997236, 3.1613073339995026, 3.149250230000689, 3.1933716659987113, 3.181643628000529]
_delorean 3.508655527999508 [3.508655527999508, 3.6782505570008652, 3.590653953999208, 3.670900126999186, 3.5859112749985798]

systemcatch avatar Dec 02 '21 16:12 systemcatch

We should probably add descriptions to each benchmark and work on graphing the results.

jadchaar avatar Dec 03 '21 02:12 jadchaar

We should probably add descriptions to each benchmark and work on graphing the results.

Absolutely, we also need to check this is actually doing fair comparisons as well. Looks like parse/now/fromtimestamp are where arrow is slower.

systemcatch avatar Dec 03 '21 15:12 systemcatch