iris
iris copied to clipboard
SciTools
Enable factory references to create new dimensions on load.
Now updated with prototype loading-control object NOTE To-dos at least :
- no proper tests yet
- work needed to implement the loading policy in load_cubes/load_cube
- fix crashing plot tests (now skipped)
@trexfeathers I think the user experience is almost there. Seems like a good compromise that still holds the user's hand as much as possible (they don't write the code themselves), while still offering more customisation than currently.
I'm not providing additional docstrings yet, but it's clearly needed (if not a userguide section?) Likewise what you already said : the new functionality badly needs usage examples -- on the TODO list
@trexfeathers The developer experience feels very complicated. I think this is worth some attention, since the developer experience around loading is already too complicated and I have noticed can be a big barrier for less experienced people. Afraid I don't have any immediate suggestions, only that it's worth some time.
You're totally right, and I'm currently planning to remove all the pieces here to a single sub-module "iris.io.loading", which I think will help a bit.
I have a draft of that, and it gathers the _CubeFilterCollection and _CubeFilter classes, supporting routines "_generate_cubes" and "_load_collection", all the new "policy" code and all the load_xxx functions.
Along with that, I am considering removing the _CubeFilterCollection and _CubeFilter classes altogether as I think they may obscure rather than help.
However, I want to add some proper testcases before I attempt a lift-and-shift, as otherwise some breakages might go unnoticed.
Current best ideas for a "new control strategy"
Following offline discussion wuith @trexfeathers
Just 3 control keywords:
-
support_multiple_references= True/False- (as before, but now implies the old
multiref_triggers_concatenate=Truebehaviour)
- (as before, but now implies the old
-
merge_concatenate_sequence= "m" / "c" / "mc" / "cm" -
repeat_until_no_changes= True/False
Plus...
- combine settings in a simple dictionary or namedtuple
- use simple class constants for common choices
- make
support_multiple_referencesTrue by default, so it ..- includes the triggering of a final concatenate
- delivers required solution for the key new cases (i.e. time-dependent hybrid height)
- delivers existing result for 99% of existing cases
- for 1% of broken legacy loads, can be easily forced to "old behaviour"
Progress Update 2024-10-14 :
For now I'm deferring the user-API improvements detailed above.
The plan now is roughly :
- [x] fix the existing functionality
- note : latest commit includes changes to the 'unique' keyword in merges, which ought to fix some of the outstanding test failures
- [ ] add explicit tests for various loading testcases, including multiple-references ones
- [ ] fix the outstanding test failures + resolve/reinstate skipped tests (i.e. the plotting crashes)
- [ ] move all the disparate moving parts into a "iris.io.loading" submodule
- [x] simplify the user API
- [ ] ? fix default loading policy to new-style ? (and fix any tests it breaks)
- [ ] add new user docstrings + whatsnew describing changes
Codecov Report
Attention: Patch coverage is 95.53571% with 5 lines in your changes missing coverage. Please review.
Project coverage is 89.86%. Comparing base (
e892fd1) to head (e5b4ce6).
| Files with missing lines | Patch % | Lines |
|---|---|---|
| lib/iris/__init__.py | 93.33% | 4 Missing and 1 partial :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## FEATURE_load_policy #6168 +/- ##
=======================================================
+ Coverage 89.82% 89.86% +0.04%
=======================================================
Files 88 88
Lines 23185 23277 +92
Branches 4314 4333 +19
=======================================================
+ Hits 20825 20917 +92
- Misses 1628 1629 +1
+ Partials 732 731 -1
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
UPDATED Progress Update 2024-10-23 :
I'm now targetting a feature branch, so we can review the work in stages before committing to main.
The remaining planned work is :
- [ ] move all the disparate moving parts into a "iris.io.loading" submodule
- add tests
- [x] tests for LoadPolicy behaviour (init, set + context methods)
- [ ] tests for new "combine_cubes" user routine
- (aside : should this maybe be cast as a Cubelist method, like merge + concatenate ?)
- [ ] ?possibly? separate tests for how rules.py now handles multiple references
- TBH I think this may be adequately proved in the integ
- [ ] ?possible? iris interface extensions
- [ ] support
options=None, **kwargsto set policy in irisload/load_cube/load_cubes - [ ] support
CubeList.combine()andcombine_cube()methods callingiris.combine_cubes
- [ ] support
- [x] reinstate + fixes skipped tests (i.e. the plotting crashes)
- [x] change actual default loading policy from "legacy" to "default" -- and fix any tests which that breaks
- [x] add whatsnew
Ok @stephenworsley I think I have covered the outstanding.
Please check against comments + resolve if appropriate.
Let me know if+when you have more points to raise -- or not !
Status 2024-10-23 @1615 :
Essentials from previous comments still to resolve :
- [x] explain meaning of reference fields in LoadPolicy docs
- [x] adjust LoadPolicy.context examples
- [x] remove spurious "policy" arg in load function
We think that unresolved comments by @trexfeathers are all either now done, or not important to fix
Hi @stephenworsley thanks for all your patience ! I think I have done everything I was planning now, and I think it addresses all the outstanding review queries. If not drop me a line - I'll check in sometime tomorrow morning, and can put a little time into it if that will get it finished off.
:stopwatch: Performance Benchmark Report: 58decfdc
Performance shifts
| Change | Before [e892fd19] | After [58decfdc] | Ratio | Benchmark (Parameter) |
|----------|----------------------|---------------------|---------|---------------------------------------------------------|
| + | 232±2μs | 294±4μs | 1.27 | import_iris.Iris.time_iris |
| + | 4.24±0.02ms | 5.11±0.07ms | 1.21 | load.LoadAndRealise.time_load((50, 50, 2), False, 'FF') |
| + | 48.8±0.6ms | 58.6±1ms | 1.2 | load.ugrid.BasicLoading.time_load_file(200000) |
Full benchmark results
Benchmarks that have stayed the same:
| Change | Before [e892fd19] | After [58decfdc] | Ratio | Benchmark (Parameter) |
|----------|----------------------|---------------------|---------|---------------------------------------------------------------------------------------------|
| | 62.4±1ms | 61.1±1ms | 0.98 | aggregate_collapse.Aggregation.time_aggregated_by_COUNT(False) |
| | 62.5±0.7ms | 63.1±1ms | 1.01 | aggregate_collapse.Aggregation.time_aggregated_by_COUNT(True) |
| | 229±2ms | 231±3ms | 1.01 | aggregate_collapse.Aggregation.time_aggregated_by_FAST_PERCENTILE(False) |
| | 231±3ms | 231±2ms | 1.00 | aggregate_collapse.Aggregation.time_aggregated_by_FAST_PERCENTILE(True) |
| | 39.9±0.4ms | 40.3±0.9ms | 1.01 | aggregate_collapse.Aggregation.time_aggregated_by_GMEAN(False) |
| | 41.1±0.4ms | 40.8±0.5ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_GMEAN(True) |
| | 40.0±0.5ms | 40.3±0.5ms | 1.01 | aggregate_collapse.Aggregation.time_aggregated_by_HMEAN(False) |
| | 41.4±0.5ms | 41.0±0.3ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_HMEAN(True) |
| | 53.8±0.8ms | 53.3±0.9ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_MAX(False) |
| | 53.8±1ms | 53.7±0.7ms | 1.00 | aggregate_collapse.Aggregation.time_aggregated_by_MAX(True) |
| | 140±2ms | 139±2ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_MAX_RUN(False) |
| | 141±1ms | 141±2ms | 1.00 | aggregate_collapse.Aggregation.time_aggregated_by_MAX_RUN(True) |
| | 59.3±0.8ms | 58.0±1ms | 0.98 | aggregate_collapse.Aggregation.time_aggregated_by_MEAN(False) |
| | 59.2±0.8ms | 59.0±1ms | 1.00 | aggregate_collapse.Aggregation.time_aggregated_by_MEAN(True) |
| | 39.6±0.3ms | 40.0±0.4ms | 1.01 | aggregate_collapse.Aggregation.time_aggregated_by_MEDIAN(False) |
| | 40.5±0.3ms | 40.8±0.4ms | 1.01 | aggregate_collapse.Aggregation.time_aggregated_by_MEDIAN(True) |
| | 53.5±0.7ms | 53.2±1ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_MIN(False) |
| | 54.2±1ms | 54.9±1ms | 1.01 | aggregate_collapse.Aggregation.time_aggregated_by_MIN(True) |
| | 1.35±0.01s | 1.36±0.02s | 1.01 | aggregate_collapse.Aggregation.time_aggregated_by_PEAK(False) |
| | 1.34±0.01s | 1.35±0.02s | 1.00 | aggregate_collapse.Aggregation.time_aggregated_by_PEAK(True) |
| | 716±20ms | 709±20ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_PERCENTILE(False) |
| | 712±10ms | 714±10ms | 1.00 | aggregate_collapse.Aggregation.time_aggregated_by_PERCENTILE(True) |
| | 38.6±0.4ms | 38.1±0.4ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_PROPORTION(False) |
| | 39.3±0.4ms | 39.1±0.6ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_PROPORTION(True) |
| | 71.6±1ms | 71.8±0.9ms | 1.00 | aggregate_collapse.Aggregation.time_aggregated_by_RMS(False) |
| | 72.2±1ms | 72.8±1ms | 1.01 | aggregate_collapse.Aggregation.time_aggregated_by_RMS(True) |
| | 73.6±0.7ms | 73.5±0.6ms | 1.00 | aggregate_collapse.Aggregation.time_aggregated_by_STD_DEV(False) |
| | 75.0±1ms | 74.0±1ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_STD_DEV(True) |
| | 68.5±0.6ms | 68.1±0.8ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_VARIANCE(False) |
| | 70.0±1ms | 69.1±0.9ms | 0.99 | aggregate_collapse.Aggregation.time_aggregated_by_VARIANCE(True) |
| | 26.8±0.3ms | 27.1±0.3ms | 1.01 | aggregate_collapse.Aggregation.time_collapsed_by_COUNT(False) |
| | 31.2±0.3ms | 31.1±0.4ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_COUNT(True) |
| | 159±2ms | 159±1ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_FAST_PERCENTILE(False) |
| | 174±2ms | 176±2ms | 1.01 | aggregate_collapse.Aggregation.time_collapsed_by_FAST_PERCENTILE(True) |
| | 25.0±0.3ms | 24.9±0.3ms | 0.99 | aggregate_collapse.Aggregation.time_collapsed_by_GMEAN(False) |
| | 29.4±0.3ms | 29.4±0.2ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_GMEAN(True) |
| | 24.9±0.3ms | 25.1±0.4ms | 1.01 | aggregate_collapse.Aggregation.time_collapsed_by_HMEAN(False) |
| | 29.1±0.4ms | 29.1±0.3ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_HMEAN(True) |
| | 25.6±0.2ms | 26.3±0.5ms | 1.03 | aggregate_collapse.Aggregation.time_collapsed_by_MAX(False) |
| | 29.9±0.3ms | 29.8±0.8ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_MAX(True) |
| | 39.3±0.6ms | 39.2±0.7ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_MAX_RUN(False) |
| | 43.1±0.9ms | 43.9±0.6ms | 1.02 | aggregate_collapse.Aggregation.time_collapsed_by_MAX_RUN(True) |
| | 26.2±0.5ms | 25.8±0.4ms | 0.99 | aggregate_collapse.Aggregation.time_collapsed_by_MEAN(False) |
| | 30.5±0.3ms | 30.4±0.3ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_MEAN(True) |
| | 25.3±0.2ms | 25.7±0.5ms | 1.01 | aggregate_collapse.Aggregation.time_collapsed_by_MEDIAN(False) |
| | 30.3±0.5ms | 30.0±0.5ms | 0.99 | aggregate_collapse.Aggregation.time_collapsed_by_MEDIAN(True) |
| | 25.9±0.6ms | 25.6±0.4ms | 0.99 | aggregate_collapse.Aggregation.time_collapsed_by_MIN(False) |
| | 29.8±0.3ms | 29.5±0.4ms | 0.99 | aggregate_collapse.Aggregation.time_collapsed_by_MIN(True) |
| | 587±3ms | 583±9ms | 0.99 | aggregate_collapse.Aggregation.time_collapsed_by_PEAK(False) |
| | 589±4ms | 588±4ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_PEAK(True) |
| | 178±2ms | 178±4ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_PERCENTILE(False) |
| | 199±3ms | 196±2ms | 0.99 | aggregate_collapse.Aggregation.time_collapsed_by_PERCENTILE(True) |
| | 24.8±0.6ms | 24.2±0.4ms | 0.98 | aggregate_collapse.Aggregation.time_collapsed_by_PROPORTION(False) |
| | 29.1±0.5ms | 29.1±0.5ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_PROPORTION(True) |
| | 28.7±0.3ms | 28.6±0.5ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_RMS(False) |
| | 32.9±0.4ms | 33.1±0.5ms | 1.01 | aggregate_collapse.Aggregation.time_collapsed_by_RMS(True) |
| | 27.9±0.5ms | 28.6±0.2ms | 1.02 | aggregate_collapse.Aggregation.time_collapsed_by_STD_DEV(False) |
| | 32.8±0.4ms | 32.6±0.5ms | 0.99 | aggregate_collapse.Aggregation.time_collapsed_by_STD_DEV(True) |
| | 27.8±0.5ms | 27.6±0.2ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_VARIANCE(False) |
| | 31.7±0.5ms | 31.8±0.4ms | 1.00 | aggregate_collapse.Aggregation.time_collapsed_by_VARIANCE(True) |
| | 97.4±2ms | 97.2±1ms | 1.00 | aggregate_collapse.WeightedAggregation.time_w_aggregated_by_MEAN(False) |
| | 98.3±1ms | 96.9±0.9ms | 0.99 | aggregate_collapse.WeightedAggregation.time_w_aggregated_by_MEAN(True) |
| | 112±1ms | 112±2ms | 0.99 | aggregate_collapse.WeightedAggregation.time_w_aggregated_by_RMS(False) |
| | 112±1ms | 112±1ms | 1.00 | aggregate_collapse.WeightedAggregation.time_w_aggregated_by_RMS(True) |
| | 66.7±0.6ms | 66.3±0.6ms | 0.99 | aggregate_collapse.WeightedAggregation.time_w_aggregated_by_SUM(False) |
| | 66.5±1ms | 67.5±0.8ms | 1.02 | aggregate_collapse.WeightedAggregation.time_w_aggregated_by_SUM(True) |
| | 32.4±0.5ms | 32.6±0.8ms | 1.01 | aggregate_collapse.WeightedAggregation.time_w_collapsed_by_MEAN(False) |
| | 37.1±0.6ms | 36.8±0.8ms | 0.99 | aggregate_collapse.WeightedAggregation.time_w_collapsed_by_MEAN(True) |
| | 34.6±0.9ms | 34.7±0.7ms | 1.00 | aggregate_collapse.WeightedAggregation.time_w_collapsed_by_RMS(False) |
| | 39.1±0.8ms | 39.0±0.7ms | 1.00 | aggregate_collapse.WeightedAggregation.time_w_collapsed_by_RMS(True) |
| | 27.9±0.3ms | 28.0±0.4ms | 1.00 | aggregate_collapse.WeightedAggregation.time_w_collapsed_by_SUM(False) |
| | 32.5±0.5ms | 32.3±0.3ms | 1.00 | aggregate_collapse.WeightedAggregation.time_w_collapsed_by_SUM(True) |
| | 352±4ms | 356±5ms | 1.01 | aggregate_collapse.WeightedAggregation.time_w_collapsed_by_WPERCENTILE(False) |
| | 374±2ms | 376±6ms | 1.01 | aggregate_collapse.WeightedAggregation.time_w_collapsed_by_WPERCENTILE(True) |
| | 1.10±0.01ms | 1.09±0.01ms | 1.00 | cube.CubeCreation.time_create(False, 'construct') |
| | 394±3μs | 394±3μs | 1.00 | cube.CubeCreation.time_create(False, 'instantiate') |
| | 941±6μs | 941±9μs | 1.00 | cube.CubeCreation.time_create(True, 'construct') |
| | 577±6μs | 573±6μs | 0.99 | cube.CubeCreation.time_create(True, 'instantiate') |
| | 254±3ms | 251±4ms | 0.99 | cube.CubeEquality.time_equality(False, False, 'all_equal') |
| | 132±1ms | 132±4ms | 1.00 | cube.CubeEquality.time_equality(False, False, 'coord_inequality') |
| | 263±3ms | 266±4ms | 1.01 | cube.CubeEquality.time_equality(False, False, 'data_inequality') |
| | 16.5±0.1μs | 16.4±0.1μs | 0.99 | cube.CubeEquality.time_equality(False, False, 'metadata_inequality') |
| | 362±5ms | 364±5ms | 1.01 | cube.CubeEquality.time_equality(False, True, 'all_equal') |
| | 244±2ms | 245±3ms | 1.01 | cube.CubeEquality.time_equality(False, True, 'coord_inequality') |
| | 372±6ms | 372±6ms | 1.00 | cube.CubeEquality.time_equality(False, True, 'data_inequality') |
| | 16.7±0.2μs | 17.0±0.2μs | 1.02 | cube.CubeEquality.time_equality(False, True, 'metadata_inequality') |
| | 252±4ms | 253±3ms | 1.00 | cube.CubeEquality.time_equality(True, False, 'all_equal') |
| | 132±3ms | 133±0.9ms | 1.00 | cube.CubeEquality.time_equality(True, False, 'coord_inequality') |
| | 263±4ms | 265±3ms | 1.01 | cube.CubeEquality.time_equality(True, False, 'data_inequality') |
| | 53.3±0.6μs | 52.5±0.4μs | 0.99 | cube.CubeEquality.time_equality(True, False, 'metadata_inequality') |
| | 357±6ms | 362±4ms | 1.02 | cube.CubeEquality.time_equality(True, True, 'all_equal') |
| | 244±2ms | 244±3ms | 1.00 | cube.CubeEquality.time_equality(True, True, 'coord_inequality') |
| | 376±6ms | 374±3ms | 1.00 | cube.CubeEquality.time_equality(True, True, 'data_inequality') |
| | 54.7±0.5μs | 55.0±0.9μs | 1.00 | cube.CubeEquality.time_equality(True, True, 'metadata_inequality') |
| | 798±6μs | 799±6μs | 1.00 | import_iris.Iris.time__concatenate |
| | 187±4μs | 187±2μs | 1.00 | import_iris.Iris.time__constraints |
| | 110±1μs | 112±1μs | 1.02 | import_iris.Iris.time__data_manager |
| | 95.9±0.6μs | 95.2±0.7μs | 0.99 | import_iris.Iris.time__deprecation |
| | 137±2μs | 135±2μs | 0.99 | import_iris.Iris.time__lazy_data |
| | 927±9μs | 920±8μs | 0.99 | import_iris.Iris.time__merge |
| | 77.4±0.4μs | 77.3±0.6μs | 1.00 | import_iris.Iris.time__representation |
| | 612±8μs | 610±10μs | 1.00 | import_iris.Iris.time_analysis |
| | 139±2μs | 144±2μs | 1.04 | import_iris.Iris.time_analysis__area_weighted |
| | 111±1μs | 111±1μs | 1.00 | import_iris.Iris.time_analysis__grid_angles |
| | 248±3μs | 253±4μs | 1.02 | import_iris.Iris.time_analysis__interpolation |
| | 190±3μs | 190±2μs | 1.00 | import_iris.Iris.time_analysis__regrid |
| | 115±1μs | 113±2μs | 0.99 | import_iris.Iris.time_analysis__scipy_interpolate |
| | 140±0.8μs | 140±3μs | 1.00 | import_iris.Iris.time_analysis_calculus |
| | 336±3μs | 331±3μs | 0.99 | import_iris.Iris.time_analysis_cartography |
| | 95.9±1μs | 94.3±0.9μs | 0.98 | import_iris.Iris.time_analysis_geomerty |
| | 226±6μs | 221±2μs | 0.98 | import_iris.Iris.time_analysis_maths |
| | 99.5±1μs | 97.6±0.6μs | 0.98 | import_iris.Iris.time_analysis_stats |
| | 182±2μs | 176±4μs | 0.97 | import_iris.Iris.time_analysis_trajectory |
| | 308±2μs | 319±7μs | 1.04 | import_iris.Iris.time_aux_factory |
| | 84.9±1μs | 84.0±0.9μs | 0.99 | import_iris.Iris.time_common |
| | 167±4μs | 164±1μs | 0.98 | import_iris.Iris.time_common_lenient |
| | 1.36±0.02ms | 1.36±0.02ms | 1.00 | import_iris.Iris.time_common_metadata |
| | 140±2μs | 139±2μs | 0.99 | import_iris.Iris.time_common_mixin |
| | 1.21±0.02ms | 1.21±0.02ms | 1.00 | import_iris.Iris.time_common_resolve |
| | 201±1μs | 201±3μs | 1.00 | import_iris.Iris.time_config |
| | 125±0.8μs | 124±2μs | 0.99 | import_iris.Iris.time_coord_categorisation |
| | 378±8μs | 387±20μs | 1.02 | import_iris.Iris.time_coord_systems |
| | 787±20μs | 789±20μs | 1.00 | import_iris.Iris.time_coords |
| | 699±10μs | 684±30μs | 0.98 | import_iris.Iris.time_cube |
| | 228±2μs | 231±4μs | 1.01 | import_iris.Iris.time_exceptions |
| | 77.7±0.6μs | 77.8±0.7μs | 1.00 | import_iris.Iris.time_experimental |
| | 186±1μs | 185±0.6μs | 1.00 | import_iris.Iris.time_fileformats |
| | 253±3μs | 259±6μs | 1.02 | import_iris.Iris.time_fileformats__ff |
| | 2.85±0.1ms | 2.76±0.02ms | 0.97 | import_iris.Iris.time_fileformats__ff_cross_references |
| | 79.2±1μs | 79.7±0.6μs | 1.01 | import_iris.Iris.time_fileformats__pp_lbproc_pairs |
| | 115±0.6μs | 116±2μs | 1.01 | import_iris.Iris.time_fileformats_abf |
| | 416±5μs | 421±8μs | 1.01 | import_iris.Iris.time_fileformats_cf |
| | 5.54±0.1ms | 5.48±0.2ms | 0.99 | import_iris.Iris.time_fileformats_dot |
| | 75.9±0.9μs | 74.9±1μs | 0.99 | import_iris.Iris.time_fileformats_name |
| | 263±6μs | 261±2μs | 0.99 | import_iris.Iris.time_fileformats_name_loaders |
| | 120±1μs | 120±2μs | 1.00 | import_iris.Iris.time_fileformats_netcdf |
| | 125±1μs | 124±2μs | 1.00 | import_iris.Iris.time_fileformats_nimrod |
| | 213±2μs | 216±4μs | 1.01 | import_iris.Iris.time_fileformats_nimrod_load_rules |
| | 792±9μs | 790±6μs | 1.00 | import_iris.Iris.time_fileformats_pp |
| | 183±3μs | 184±2μs | 1.00 | import_iris.Iris.time_fileformats_pp_load_rules |
| | 136±0.5μs | 135±0.9μs | 0.99 | import_iris.Iris.time_fileformats_pp_save_rules |
| | 525±6μs | 550±4μs | 1.05 | import_iris.Iris.time_fileformats_rules |
| | 224±4μs | 223±2μs | 1.00 | import_iris.Iris.time_fileformats_structured_array_identification |
| | 84.7±0.7μs | 83.6±1μs | 0.99 | import_iris.Iris.time_fileformats_um |
| | 162±1μs | 164±2μs | 1.01 | import_iris.Iris.time_fileformats_um__fast_load |
| | 143±0.5μs | 140±2μs | 0.98 | import_iris.Iris.time_fileformats_um__fast_load_structured_fields |
| | 77.0±0.2μs | 76.5±0.5μs | 0.99 | import_iris.Iris.time_fileformats_um__ff_replacement |
| | 83.2±0.6μs | 83.2±0.9μs | 1.00 | import_iris.Iris.time_fileformats_um__optimal_array_structuring |
| | 992±9μs | 998±5μs | 1.01 | import_iris.Iris.time_fileformats_um_cf_map |
| | 138±3μs | 137±1μs | 0.99 | import_iris.Iris.time_io |
| | 177±3μs | 177±3μs | 1.00 | import_iris.Iris.time_io_format_picker |
| | 129±2μs | 130±2μs | 1.01 | import_iris.Iris.time_iterate |
| | 8.57±0.1ms | 8.55±0.2ms | 1.00 | import_iris.Iris.time_palette |
| | 1.91±0.01ms | 1.91±0.01ms | 1.00 | import_iris.Iris.time_plot |
| | 106±0.6μs | 105±1μs | 0.99 | import_iris.Iris.time_quickplot |
| | 2.24±0.07ms | 2.23±0.2ms | 1.00 | import_iris.Iris.time_std_names |
| | 1.82±0.01ms | 1.80±0.02ms | 0.99 | import_iris.Iris.time_symbols |
| | 99.0±0.3ms | 99.0±0.9ms | 1.00 | import_iris.Iris.time_tests |
| | 257±4μs | 256±6μs | 0.99 | import_iris.Iris.time_third_party_cartopy |
| | 4.85±0.05ms | 4.80±0.05ms | 0.99 | import_iris.Iris.time_third_party_cf_units |
| | 121±2μs | 119±0.7μs | 0.98 | import_iris.Iris.time_third_party_cftime |
| | 2.89±0.04ms | 2.88±0.02ms | 1.00 | import_iris.Iris.time_third_party_matplotlib |
| | 1.07±0.01ms | 1.07±0.01ms | 1.00 | import_iris.Iris.time_third_party_numpy |
| | 171±2μs | 170±1μs | 1.00 | import_iris.Iris.time_third_party_scipy |
| | 103±2μs | 102±1μs | 0.99 | import_iris.Iris.time_time |
| | 327±4μs | 329±5μs | 1.01 | import_iris.Iris.time_util |
| | 71.6±0.8μs | 69.9±0.8μs | 0.98 | iterate.IZip.time_izip |
| | 8.99±0.04ms | 9.81±0.2ms | 1.09 | load.LoadAndRealise.time_load((1280, 960, 5), False, 'FF') |
| | 26.6±0.3ms | 27.6±0.2ms | 1.04 | load.LoadAndRealise.time_load((1280, 960, 5), False, 'NetCDF') |
| | 9.78±0.05ms | 10.6±0.09ms | 1.08 | load.LoadAndRealise.time_load((1280, 960, 5), False, 'PP') |
| | 8.99±0.1ms | 9.85±0.1ms | 1.10 | load.LoadAndRealise.time_load((1280, 960, 5), True, 'FF') |
| | 22.0±0.2ms | 23.1±0.3ms | 1.05 | load.LoadAndRealise.time_load((1280, 960, 5), True, 'NetCDF') |
| | 9.74±0.06ms | 10.8±0.3ms | 1.11 | load.LoadAndRealise.time_load((1280, 960, 5), True, 'PP') |
| | 1.53±0.01s | 1.54±0.01s | 1.00 | load.LoadAndRealise.time_load((2, 2, 1000), False, 'FF') |
| | 22.1±0.4ms | 23.8±0.6ms | 1.07 | load.LoadAndRealise.time_load((2, 2, 1000), False, 'NetCDF') |
| | 1.69±0.01s | 1.71±0.01s | 1.01 | load.LoadAndRealise.time_load((2, 2, 1000), False, 'PP') |
| | 1.53±0.01s | 1.54±0.01s | 1.00 | load.LoadAndRealise.time_load((2, 2, 1000), True, 'FF') |
| | 22.3±0.3ms | 23.3±0.2ms | 1.05 | load.LoadAndRealise.time_load((2, 2, 1000), True, 'NetCDF') |
| | 1.69±0.01s | 1.70±0.02s | 1.01 | load.LoadAndRealise.time_load((2, 2, 1000), True, 'PP') |
| | 20.8±0.3ms | 21.4±0.1ms | 1.03 | load.LoadAndRealise.time_load((50, 50, 2), False, 'NetCDF') |
| | 4.64±0.06ms | 5.31±0.06ms | 1.14 | load.LoadAndRealise.time_load((50, 50, 2), False, 'PP') |
| | 4.24±0.03ms | 5.05±0.05ms | 1.19 | load.LoadAndRealise.time_load((50, 50, 2), True, 'FF') |
| | 20.7±0.2ms | 21.6±0.2ms | 1.04 | load.LoadAndRealise.time_load((50, 50, 2), True, 'NetCDF') |
| | 4.53±0.05ms | 5.35±0.1ms | 1.18 | load.LoadAndRealise.time_load((50, 50, 2), True, 'PP') |
| | 32.9±1ms | 32.4±3ms | 0.99 | load.LoadAndRealise.time_realise((1280, 960, 5), False, 'FF') |
| | 20.9±0.4ms | 21.7±0.4ms | 1.04 | load.LoadAndRealise.time_realise((1280, 960, 5), False, 'NetCDF') |
| | 13.6±2ms | 13.7±2ms | 1.00 | load.LoadAndRealise.time_realise((1280, 960, 5), False, 'PP') |
| | 26.5±2ms | 26.6±3ms | 1.00 | load.LoadAndRealise.time_realise((1280, 960, 5), True, 'FF') |
| | 71.3±1ms | 70.9±0.9ms | 0.99 | load.LoadAndRealise.time_realise((1280, 960, 5), True, 'NetCDF') |
| | 26.7±0.7ms | 26.7±2ms | 1.00 | load.LoadAndRealise.time_realise((1280, 960, 5), True, 'PP') |
| | 503±3ms | 506±4ms | 1.01 | load.LoadAndRealise.time_realise((2, 2, 1000), False, 'FF') |
| | 3.26±0.02ms | 3.19±0.04ms | 0.98 | load.LoadAndRealise.time_realise((2, 2, 1000), False, 'NetCDF') |
| | 511±2ms | 511±3ms | 1.00 | load.LoadAndRealise.time_realise((2, 2, 1000), False, 'PP') |
| | 509±2ms | 512±3ms | 1.01 | load.LoadAndRealise.time_realise((2, 2, 1000), True, 'FF') |
| | 3.25±0.04ms | 3.21±0.09ms | 0.99 | load.LoadAndRealise.time_realise((2, 2, 1000), True, 'NetCDF') |
| | 515±5ms | 516±4ms | 1.00 | load.LoadAndRealise.time_realise((2, 2, 1000), True, 'PP') |
| | 1.80±0.08ms | 1.78±0.1ms | 0.99 | load.LoadAndRealise.time_realise((50, 50, 2), False, 'FF') |
| | 3.20±0.06ms | 3.26±0.09ms | 1.02 | load.LoadAndRealise.time_realise((50, 50, 2), False, 'NetCDF') |
| | 1.80±0.09ms | 1.84±0.07ms | 1.02 | load.LoadAndRealise.time_realise((50, 50, 2), False, 'PP') |
| | 1.77±0.1ms | 1.78±0.1ms | 1.01 | load.LoadAndRealise.time_realise((50, 50, 2), True, 'FF') |
| | 3.21±0.1ms | 3.28±0.09ms | 1.02 | load.LoadAndRealise.time_realise((50, 50, 2), True, 'NetCDF') |
| | 1.78±0.08ms | 1.82±0.05ms | 1.02 | load.LoadAndRealise.time_realise((50, 50, 2), True, 'PP') |
| | 361±4ms | 358±3ms | 0.99 | load.ManyVars.time_many_var_load |
| | 9.26±0.2ms | 9.93±0.08ms | 1.07 | load.STASHConstraint.time_stash_constraint((1280, 960, 5), 'FF') |
| | 9.99±0.1ms | 10.8±0.05ms | 1.08 | load.STASHConstraint.time_stash_constraint((1280, 960, 5), 'PP') |
| | 1.54±0.01s | 1.55±0.02s | 1.01 | load.STASHConstraint.time_stash_constraint((2, 2, 1000), 'FF') |
| | 1.71±0.01s | 1.73±0.01s | 1.01 | load.STASHConstraint.time_stash_constraint((2, 2, 1000), 'PP') |
| | 4.30±0.04ms | 5.15±0.08ms | 1.20 | load.STASHConstraint.time_stash_constraint((2, 2, 2), 'FF') |
| | 4.72±0.1ms | 5.40±0.03ms | 1.14 | load.STASHConstraint.time_stash_constraint((2, 2, 2), 'PP') |
| | 9.20±0.2ms | 8.96±0.1ms | 0.97 | load.StructuredFF.time_structured_load((1280, 960, 5), False) |
| | 5.85±0.08ms | 6.06±0.2ms | 1.04 | load.StructuredFF.time_structured_load((1280, 960, 5), True) |
| | 1.51±0.01s | 1.53±0.02s | 1.01 | load.StructuredFF.time_structured_load((2, 2, 1000), False) |
| | 536±10ms | 539±2ms | 1.01 | load.StructuredFF.time_structured_load((2, 2, 1000), True) |
| | 4.26±0.05ms | 4.27±0.1ms | 1.00 | load.StructuredFF.time_structured_load((2, 2, 2), False) |
| | 4.21±0.05ms | 4.15±0.03ms | 0.99 | load.StructuredFF.time_structured_load((2, 2, 2), True) |
| | 165±0.9ms | 164±0.5ms | 0.99 | load.TimeConstraint.time_time_constraint(20, 'FF') |
| | 24.6±0.4ms | 25.8±0.4ms | 1.05 | load.TimeConstraint.time_time_constraint(20, 'NetCDF') |
| | 182±0.9ms | 182±1ms | 1.00 | load.TimeConstraint.time_time_constraint(20, 'PP') |
| | 32.3±0.5ms | 33.1±0.3ms | 1.02 | load.TimeConstraint.time_time_constraint(3, 'FF') |
| | 24.4±0.2ms | 24.7±0.3ms | 1.02 | load.TimeConstraint.time_time_constraint(3, 'NetCDF') |
| | 34.8±0.5ms | 35.4±0.4ms | 1.02 | load.TimeConstraint.time_time_constraint(3, 'PP') |
| | 18.5±0.6ms | 20.2±0.6ms | 1.09 | load.ugrid.BasicLoading.time_load_file(1) |
| | 15.4±0.2ms | 15.4±0.4ms | 1.00 | load.ugrid.BasicLoading.time_load_mesh(1) |
| | 27.1±0.3ms | 26.7±0.8ms | 0.99 | load.ugrid.BasicLoading.time_load_mesh(200000) |
| | 18.5±0.4ms | 20.1±0.6ms | 1.08 | load.ugrid.BasicLoadingTime.time_load_file(1) |
| | 23.6±0.3ms | 24.8±0.2ms | 1.05 | load.ugrid.BasicLoadingTime.time_load_file(200000) |
| | 15.0±0.3ms | 15.5±0.3ms | 1.03 | load.ugrid.BasicLoadingTime.time_load_mesh(1) |
| | 20.0±0.3ms | 20.1±0.2ms | 1.00 | load.ugrid.BasicLoadingTime.time_load_mesh(200000) |
| | 19.3±0.5ms | 21.3±0.3ms | 1.10 | load.ugrid.Callback.time_load_file_callback(1) |
| | 59.4±1ms | 68.4±0.7ms | 1.15 | load.ugrid.Callback.time_load_file_callback(200000) |
| | 20.0±0.6ms | 21.2±0.3ms | 1.06 | load.ugrid.CallbackTime.time_load_file_callback(1) |
| | 25.5±0.3ms | 26.3±0.4ms | 1.03 | load.ugrid.CallbackTime.time_load_file_callback(200000) |
| | 3.12±0.07ms | 3.21±0.07ms | 1.03 | load.ugrid.DataRealisation.time_realise_data(10000) |
| | 5.28±0.2ms | 4.41±0.2ms | 0.84 | load.ugrid.DataRealisation.time_realise_data(200000) |
| | 40.7±1ms | 41.8±0.9ms | 1.03 | load.ugrid.DataRealisationTime.time_realise_data(10000) |
| | 799±9ms | 799±8ms | 1.00 | load.ugrid.DataRealisationTime.time_realise_data(200000) |
| | 439±4ms | 440±4ms | 1.00 | merge_concat.Concatenate.time_concatenate(False) |
| | 438±5ms | 436±4ms | 0.99 | merge_concat.Concatenate.time_concatenate(True) |
| | 109±0.3M | 109±0.3M | 1.00 | merge_concat.Concatenate.tracemalloc_concatenate(False) |
| | 109±0.3M | 109±0.3M | 1.00 | merge_concat.Concatenate.tracemalloc_concatenate(True) |
| | 61.7±0.9ms | 62.0±1ms | 1.00 | merge_concat.Merge.time_merge |
| | 760±300k | 914±300k | ~1.20 | merge_concat.Merge.tracemalloc_merge |
| | 484±3ns | 487±6ns | 1.01 | mesh.utils.regions_combine.CombineRegionsComputeRealData.time_compute_data(50) |
| | 260±3ms | 261±1ms | 1.01 | mesh.utils.regions_combine.CombineRegionsComputeRealData.time_compute_data(500) |
| | 658±3k | 658±2k | 1.00 | mesh.utils.regions_combine.CombineRegionsComputeRealData.tracemalloc_compute_data(50) |
| | 60.1±0M | 60.1±0M | 1.00 | mesh.utils.regions_combine.CombineRegionsComputeRealData.tracemalloc_compute_data(500) |
| | 17.1±0.3ms | 17.1±0.3ms | 1.00 | mesh.utils.regions_combine.CombineRegionsCreateCube.time_create_combined_cube(50) |
| | 19.3±0.4ms | 19.0±0.6ms | 0.98 | mesh.utils.regions_combine.CombineRegionsCreateCube.time_create_combined_cube(500) |
| | 503±10k | 494±10k | 0.98 | mesh.utils.regions_combine.CombineRegionsCreateCube.tracemalloc_create_combined_cube(50) |
| | 12.4±0.01M | 12.4±0.01M | 1.00 | mesh.utils.regions_combine.CombineRegionsCreateCube.tracemalloc_create_combined_cube(500) |
| | 123±1ms | 123±1ms | 1.00 | mesh.utils.regions_combine.CombineRegionsFileStreamedCalc.time_stream_file2file(50) |
| | 741±6ms | 742±7ms | 1.00 | mesh.utils.regions_combine.CombineRegionsFileStreamedCalc.time_stream_file2file(500) |
| | 1.21±0.01M | 1.21±0.05M | 1.00 | mesh.utils.regions_combine.CombineRegionsFileStreamedCalc.tracemalloc_stream_file2file(50) |
| | 96.3±0.02M | 96.3±0.04M | 1.00 | mesh.utils.regions_combine.CombineRegionsFileStreamedCalc.tracemalloc_stream_file2file(500) |
| | 80.7±1ms | 80.5±0.8ms | 1.00 | mesh.utils.regions_combine.CombineRegionsSaveData.time_save(50) |
| | 694±3ms | 694±5ms | 1.00 | mesh.utils.regions_combine.CombineRegionsSaveData.time_save(500) |
| | 1.18±0.01M | 1.18±0.01M | 1.00 | mesh.utils.regions_combine.CombineRegionsSaveData.tracemalloc_save(50) |
| | 96.2±0.01M | 96.2±0.02M | 1.00 | mesh.utils.regions_combine.CombineRegionsSaveData.tracemalloc_save(500) |
| | 2.1752849999999997 | 2.1752849999999997 | 1.00 | mesh.utils.regions_combine.CombineRegionsSaveData.track_filesize_saved(50) |
| | 216.01528499999998 | 216.01528499999998 | 1.00 | mesh.utils.regions_combine.CombineRegionsSaveData.track_filesize_saved(500) |
| | 6.72±0.08ms | 6.99±0.1ms | 1.04 | plot.AuxSort.time_aux_sort |
| | 79.9±7ms | 82.6±5ms | 1.03 | regridding.CurvilinearRegridding.time_regrid_pic |
| | 136±3M | 136±3M | 1.00 | regridding.CurvilinearRegridding.tracemalloc_regrid_pic |
| | 102±0.5ms | 102±0.9ms | 1.00 | regridding.HorizontalChunkedRegridding.time_regrid_area_w |
| | 51.4±1ms | 50.8±1ms | 0.99 | regridding.HorizontalChunkedRegridding.time_regrid_area_w_new_grid |
| | 106±0.04M | 106±0.04M | 1.00 | regridding.HorizontalChunkedRegridding.tracemalloc_regrid_area_w |
| | 147±0.02M | 147±0.02M | 1.00 | regridding.HorizontalChunkedRegridding.tracemalloc_regrid_area_w_new_grid |
| | 4.60±0.2ms | 4.67±0.09ms | 1.01 | save.NetcdfSave.time_netcdf_save_cube(50, False) |
| | 83.2±0.8ms | 84.8±1ms | 1.02 | save.NetcdfSave.time_netcdf_save_cube(50, True) |
| | 54.3±1ms | 53.4±0.5ms | 0.98 | save.NetcdfSave.time_netcdf_save_cube(600, False) |
| | 594±6ms | 598±6ms | 1.01 | save.NetcdfSave.time_netcdf_save_cube(600, True) |
| | 89.3±1ns | 90.9±0.7ns | 1.02 | save.NetcdfSave.time_netcdf_save_mesh(50, False) |
| | 65.5±0.7ms | 66.7±0.8ms | 1.02 | save.NetcdfSave.time_netcdf_save_mesh(50, True) |
| | 89.7±1ns | 88.4±1ns | 0.99 | save.NetcdfSave.time_netcdf_save_mesh(600, False) |
| | 524±5ms | 528±2ms | 1.01 | save.NetcdfSave.time_netcdf_save_mesh(600, True) |
| | 28±0.07k | 28±0.07k | 1.00 | save.NetcdfSave.tracemalloc_netcdf_save(50, False) |
| | 1.61±0.1M | 1.77±0.1M | 1.10 | save.NetcdfSave.tracemalloc_netcdf_save(50, True) |
| | 28±0.06k | 28±0.06k | 1.00 | save.NetcdfSave.tracemalloc_netcdf_save(600, False) |
| | 216±20M | 225±30M | 1.04 | save.NetcdfSave.tracemalloc_netcdf_save(600, True) |
| | 48.8±1ms | 48.5±1ms | 0.99 | stats.PearsonR.time_lazy |
| | 19.6±0.3ms | 19.4±0.2ms | 0.99 | stats.PearsonR.time_real |
| | 20±0.2M | 19.9±0.2M | 1.00 | stats.PearsonR.tracemalloc_lazy |
| | 18.4±0.01M | 18.4±0.01M | 1.00 | stats.PearsonR.tracemalloc_real |
| | 28.3±0.7ms | 28.1±1ms | 0.99 | trajectory.TrajectoryInterpolation.time_trajectory_linear |
| | 70.2±0.9ms | 70.3±0.3ms | 1.00 | trajectory.TrajectoryInterpolation.time_trajectory_nearest |
| | 23.3±0.01M | 23.3±0.01M | 1.00 | trajectory.TrajectoryInterpolation.tracemalloc_trajectory_linear |
| | 12.1±0.05M | 12.1±0.05M | 1.00 | trajectory.TrajectoryInterpolation.tracemalloc_trajectory_nearest |
Benchmarks that have got worse:
| Change | Before [e892fd19] | After [58decfdc] | Ratio | Benchmark (Parameter) |
|----------|----------------------|---------------------|---------|---------------------------------------------------------|
| + | 232±2μs | 294±4μs | 1.27 | import_iris.Iris.time_iris |
| + | 4.24±0.02ms | 5.11±0.07ms | 1.21 | load.LoadAndRealise.time_load((50, 50, 2), False, 'FF') |
| + | 48.8±0.6ms | 58.6±1ms | 1.2 | load.ugrid.BasicLoading.time_load_file(200000) |
Generated by GHA run 11514581752
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}