iris icon indicating copy to clipboard operation
iris copied to clipboard

Performance Shift(s): `44fae030`

Open github-actions[bot] opened this issue 2 years ago • 6 comments

Benchmark comparison has identified performance shifts at

  • commit 44fae030 (#4840).

Please review the report below and take corrective/congratulatory action as appropriate :slightly_smiling_face:

Performance shift report
       before           after         ratio
     [07b223ae]       [44fae030]
     <main~1>         <main>    
+       73.5±10μs         92.4±7μs     1.26  cube.MeshCoord.time_remove(10000)
+      1.78±0.01s       8.15±0.07s     4.57  load.LoadAndRealise.time_load((2, 2, 1000), False, 'FF')
+      1.93±0.01s       8.24±0.09s     4.28  load.LoadAndRealise.time_load((2, 2, 1000), False, 'PP')
+      1.79±0.01s       8.15±0.08s     4.56  load.LoadAndRealise.time_load((2, 2, 1000), True, 'FF')
+      1.93±0.01s       8.25±0.06s     4.28  load.LoadAndRealise.time_load((2, 2, 1000), True, 'PP')
+      1.73±0.07s       8.21±0.08s     4.75  load.STASHConstraint.time_stash_constraint((2, 2, 1000), 'FF')
+      1.88±0.08s       8.30±0.06s     4.41  load.STASHConstraint.time_stash_constraint((2, 2, 1000), 'PP')
+      1.77±0.01s       8.25±0.07s     4.65  load.StructuredFF.time_structured_load((2, 2, 1000), False)

Generated by GHA run 2593231305

github-actions[bot] avatar Jul 01 '22 00:07 github-actions[bot]

Well, this is a humdinger...

bjlittle avatar Jul 01 '22 08:07 bjlittle

@wjbenfold had some ideas for improving loading time for files with many fields - there was some looping that was inappropriate. I seem to remember that the proposed changes got watered down a lot, although I can't remember why. Perhaps this issue changes the landscape somewhat, could be worth another visit?

trexfeathers avatar Jul 01 '22 09:07 trexfeathers

Oof.

jamesp avatar Jul 01 '22 15:07 jamesp

How does the benchmark infra work @trexfeathers? Does it use the new environment file (python 3.10) included in this PR to generate the numbers in the after column?

It's probably worth checking whether this is an environment regression or a result of the code change.

jamesp avatar Jul 01 '22 15:07 jamesp

How does the benchmark infra work @trexfeathers? Does it use the new environment file (python 3.10) included in this PR to generate the numbers in the after column?

It's probably worth checking whether this is an environment regression or a result of the code change.

Correct. In normal circumstances this allows us to distinguish environment vs code change, since we have lockfile-specific commits. Bit difficult in this case!

trexfeathers avatar Jul 01 '22 15:07 trexfeathers

https://github.com/SciTools/iris/pull/4852#issuecomment-1175951526

trexfeathers avatar Jul 06 '22 08:07 trexfeathers