Compatibility with mpl v3.9.2 (latest)

[cvanelteren]

Continuation of #458 and #450. This PR makes it up-to-date with the latest mpl version. It passes the generate unittests from #458, I will do some finalize testing since the colormaps are the major change in the last couple of mpl versions.

[cvanelteren]

• TODO fix docs
• Note this makes use of the backwards compatibility for the colormaps. Over time these will be deprecated by mpl. I am still trying to figure out how @lukelbd uses the colormaps here, and how to adapt the code to mpl. For now everything seems to be running fine

[cvanelteren]

Just an update: a lot of things are broken, but on the bright side some things are working. I don't yet understand how to properly override the existing colormap registry to include the local maps that are added by proplot. The main things that are added by proplot (next to colormaps) are the ability to lookup without case sensitivity.

current errors

=========================================================== short test summary info ===========================================================
FAILED proplot/tests/test_1dplots.py::test_bar_vectors - TypeError: alpha must be numeric or None, not <class 'numpy.ndarray'>
FAILED proplot/tests/test_1dplots.py::test_boxplot_colors - TypeError: Axes.violinplot() got an unexpected keyword argument 'hatches'
FAILED proplot/tests/test_1dplots.py::test_boxplot_vectors - TypeError: Axes.boxplot() got an unexpected keyword argument 'hatch'
FAILED proplot/tests/test_1dplots.py::test_scatter_alpha - Failed: DID NOT WARN. No warnings of type (<class 'proplot.internals.warnings.ProplotWarning'>,) were emitted.
FAILED proplot/tests/test_2dplots.py::test_colormap_vcenter - AttributeError: PolyQuadMesh.set() got an unexpected keyword argument 'vcenter'
FAILED proplot/tests/test_2dplots.py::test_autodiverging2 - AttributeError: PolyQuadMesh.set() got an unexpected keyword argument 'vcenter'
FAILED proplot/tests/test_axes.py::test_panels_suplabels - AttributeError: CartesianAxes.set() got an unexpected keyword argument 'refwidth'
FAILED proplot/tests/test_colorbar.py::test_segmented_norm_center - TypeError: SegmentedNorm.__init__() got an unexpected keyword argument 'vcenter'
FAILED proplot/tests/test_format.py::test_ignored_keywords - AttributeError: module 'proplot.tests' has no attribute 'warns'
FAILED proplot/tests/test_format.py::test_patch_format - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_format.py::test_multi_formatting - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_format.py::test_inner_title_zorder - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_format.py::test_axes_colors - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_format.py::test_locale_formatting - locale.Error: unsupported locale setting
FAILED proplot/tests/test_integration.py::test_pint_quantities - NameError: name 'pint' is not defined
FAILED proplot/tests/test_integration.py::test_seaborn_hist - TypeError: kdeplot() takes from 0 to 1 positional arguments but 2 positional arguments (and 1 keyword-only argument) were given
FAILED proplot/tests/test_projections.py::test_aspect_ratios - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_projections.py::test_cartopy_labels - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_projections.py::test_cartopy_contours - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_projections.py::test_cartopy_manual - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_projections.py::test_projection_dicts - AttributeError: '_LonAxis' object has no attribute 'axis_name'
=========================================== 21 failed, 88 passed, 4 skipped, 60 warnings in 19.99s ============================================

[cvanelteren]

@bradyrx perhaps you could look into the errors regarding geopy. I am not too familiar with geoaxes, but the errors seem relatively straight forward if you are aware of those kind of axes.

[cvanelteren]

Starting to look good

errors left to fix

================================================================== short test summary info ===================================================================
FAILED proplot/tests/test_format.py::test_patch_format - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_format.py::test_multi_formatting - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_format.py::test_inner_title_zorder - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_format.py::test_axes_colors - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_format.py::test_locale_formatting - locale.Error: unsupported locale setting
FAILED proplot/tests/test_integration.py::test_pint_quantities - NameError: name 'pint' is not defined
FAILED proplot/tests/test_integration.py::test_seaborn_hist - TypeError: kdeplot() takes from 0 to 1 positional arguments but 2 positional arguments (and 1 keyword-only argument) were given
FAILED proplot/tests/test_projections.py::test_aspect_ratios - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_projections.py::test_cartopy_labels - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_projections.py::test_cartopy_contours - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_projections.py::test_cartopy_manual - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_projections.py::test_projection_dicts - AttributeError: '_LonAxis' object has no attribute 'axis_name'
FAILED proplot/tests/test_subplots.py::test_reference_aspect - assert False
=================================================== 13 failed, 92 passed, 7 skipped, 66 warnings in 35.09s ================================================

[cvanelteren]

For those interested in giving it a spin, the day to day running works fine and I don't run into issues -- that said I do not use geoaxes. Install by cloning the fork and installing it through there. I am Running python 3.12.

[didi-maru]

@cvanelteren tank you! I tried to make geoaxes work with your fork, it seems that very few changes were necessary (see https://github.com/didi-maru/proplot/commit/32bf3ddb0fd5c082d2e53a8df6898f11ce4c5732) I tested with Python 3.10 mpl 3.9.1 and cartopy 0.23.0.

tests results

============================================================================================== short test summary info ===============================================================================================
FAILED proplot/tests/test_1dplots.py::test_boxplot_vectors - ValueError: operands could not be broadcast together with shapes (10,) (20,)
FAILED proplot/tests/test_format.py::test_patch_format - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_format.py::test_multi_formatting - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_format.py::test_inner_title_zorder - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_format.py::test_locale_formatting - locale.Error: unsupported locale setting
FAILED proplot/tests/test_imshow.py::test_inbounds_data - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_imshow.py::test_colorbar - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_integration.py::test_pint_quantities - NameError: name 'pint' is not defined
FAILED proplot/tests/test_integration.py::test_keep_guide_labels - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_integration.py::test_seaborn_swarmplot - KeyError: "'_magma_copy_r' is not a known colormap name"
FAILED proplot/tests/test_integration.py::test_seaborn_hist - TypeError: kdeplot() takes from 0 to 1 positional arguments but 2 positional arguments (and 1 keyword-only argument) were given
FAILED proplot/tests/test_subplots.py::test_reference_aspect - assert np.False_
============================================================================== 12 failed, 93 passed, 7 skipped, 245 warnings in 33.43s ===============================================================================

[cvanelteren]

@didi-maru Nice team work! Just added it to this PR ;-)! On wards and upwards!

[cvanelteren]

All tests pass :partying_face:

There is still some work to do however.

• We cannot update numpy to 2.0 due to a dask issue that some of the tests (and I am guessing cartopy users require).
• There are some depecration warnings I would like to clean up before submitting
• Looking through the generated baselines there are some funky oddities that would require some closer inspection but it is looking good ;-)!
• Edit I also have to check why the read the docs are creating errors. At least, I think, the majority of work is done. Pinging @bradyrx for pre-celebrations.

[cvanelteren]

Note to future me:

• Check some of the ~~imshow~~ pcolor baselines, they have extra white space that shouldn't be there (~~could be the tests~~)
• Check the ticker, some of the tickers are spaced out too tightly (could be the tests).
• Replace some of the statistical tests with the ones from the docs (there is no reference currently how they should look and they look odd).
• Look into the cmap remapping. This is currently skipped, and does not work as advertised. It is related I think to the separate colormapregistry and colormap database (still don't understand the order of loading).

[riley-brady]

Awesome job so far!! Am I understanding that this is the focus PR and includes all of #458 and more?

I would say to add an explicit automated Github Action to this PR to run all of this unit tests so it is appended to the PR. Is that something you have experience with? Happy to add this if needed.

Would think we'd want to add

• Standard testing for proplot install on various architectures (e.g. https://github.com/pangeo-data/climpred/blob/main/.github/workflows/climpred_installs.yml)
• Variety of unit testing, using a matrix of matplotlib, cartopy, and python installs. E.g. https://github.com/pangeo-data/climpred/blob/main/.github/workflows/climpred_testing.yml or better examples out there.
• Some sort of nightly or weekly cron job that runs upstream against the latest matplotlib and cartopy. I think a lot of the open source standard packages have examples of these.

[riley-brady]

See https://github.com/readthedocs/readthedocs.org/issues/11173 potentially for readthedocs breaks.

Another level of testing will be to add docs testing so the docs examples aren't out of line with where the package goes. Probably overkill to add that to this PR. More important is unit testing working and that proplot works with modern package versions.

[cvanelteren]

This PR is intended to do two things

• To add unittests to proplot
• To make it up-to-date with the latest mpl

In hindsight I should have separated these more cleanly. I can have a look if it is still possible to separate them so we can also have backwards compatible unittest from 3.4.2 onwards.

I am going to have a look at cmasher how they handle the new colormap registration as this is still not working correctly. Second, there are some functions that produce odd results (pcolormesh for example). The axes objects also loose there ability to slice in a 2D manner. Lastly, the colorbars are not aligned with the axis object when viewing it in qt5 for example, but they are aligned when saving the figure, guessing there is some tight_layout shenanigans going on. Other than those, everything has been running smoothly, I am daily driving this to see if I can spot some obvious errors during my day-to-day.

[cvanelteren]

I will look into the points you addressed @riley-brady when the above is fixed. Then I would feel more comfortable that proplot is compatible with the latest mpl.

[cvanelteren]

Note to self: Colormasher just uses the default register functions (see https://github.com/1313e/CMasher/blob/b5b4ed6d9ce7a1b2811664afd8e1ffe836603e82/cmasher/utils.py#L1391C1-L1403C57)

[cvanelteren]

Not really sure what causes the differences with pcolor/pcolormesh. For example test_auto_diverging1 creates:

but when recreating in a separate py file I get the plot

so somehow when exporting with pytest mpl it generates this difference that I cannot reproduce on the proplot side.

[cvanelteren]

Note to self: all tests pass at the moment. The doctests, however, still create some errors. The Colormap registration is now properly handled.

[cvanelteren]

@riley-brady see comment above. Should we ping Luke? I am pretty confident that I made proplot compatible with 3.9.1 but the doc tests currently fail.

[riley-brady]

Great news! I sent him a text yesterday about this and pointing out the PR number. You've done awesome work here.

[cvanelteren]

Converting this back to draft; I need to look at the colormap again, and preserve the original names of the lookup instead of forcing the maps.

[cvanelteren]

@riley-brady I got proplot to build again, but running into some env issues; will figure it out eventually. Locally some doctests fail in the geoaxes. They will needs some TLC regarding the keywords, other than that things are looking pretty good and close to done.

[riley-brady]

Thanks for continuing to push on this!

[cvanelteren]

I am still trying to figure out what causes the issue with the theme. I temporarily set it to the default theme and then all the tests pass including the doctests. :bell: Pinging @lukelbd :bell:

[riley-brady]

I think if it's just a readthedocs theme I wouldn't worry about it for now. If all of your unit testing is passing and all looks good here, I think we should just advertise across the issues and PRs for people to install:

pip install git+https://github.com/proplot-dev/proplot.git@refs/pull/459/head

until @lukelbd merges this into a proper release.

Incredible work! I haven't gotten to test in our repos just yet. Really exciting though

[cvanelteren]

The code is now completely compatible. The docs are getting built properly (locally) but hangs in the env from read the docs. This causes a timeout. Locally that step also takes long, but eventually finishes. I have the suspicion this is causes by the projections tested at the end.

[cvanelteren]

Ready for merge

[riley-brady]

Setting up environment for testing, with a bunch of bleeding edge modern package versions with python 3.11:

mamba create -c conda-forge -n pplot_test python=3.11 matplotlib cartopy numpy pandas xarray

Trying to install from most recent official proplot release:

conda activate pplot_test
mamba install -c conda-forge proplot

The following package could not be installed
└─ proplot is installable and it requires
   ├─ matplotlib <3.5.0a0  with the potential options
   │  ├─ matplotlib [3.3.2|3.3.3|...|3.4.3] would require
   │  │  └─ python >=3.8,<3.9.0a0 , which can be installed;
   │  ├─ matplotlib [3.3.2|3.3.3|...|3.4.3] would require
   │  │  └─ python >=3.9,<3.10.0a0 , which can be installed;
   │  └─ matplotlib 3.4.3 would require
   │     └─ python >=3.10,<3.11.0a0 , which can be installed;
   └─ matplotlib-base <3.5.0a0  with the potential options
      ├─ matplotlib-base [3.3.2|3.3.3|...|3.4.3] would require
      │  └─ python >=3.8,<3.9.0a0 , which can be installed;
      ├─ matplotlib-base [3.3.2|3.3.3|...|3.4.3] would require
      │  └─ python >=3.9,<3.10.0a0 , which can be installed;
      └─ matplotlib-base 3.4.3 would require
         └─ python >=3.10,<3.11.0a0 , which can be installed.

Now with your branch:

pip install git+https://github.com/proplot-dev/proplot.git@refs/pull/459/head

Install successful!! 🎉

Quick test:

import xarray as xr
import numpy as np
import proplot as plot

xr.DataArray(np.random.random((50, 50)), dims=['x', 'y']).plot()

[riley-brady]

Awesome job @cvanelteren!! Excited for @lukelbd to merge this in when he has time. In the meantime will broadcast on old issues and PRs to install this way.

[Holmgren825]

Hey, this is wonderful! Very happy to be able to update soon. I tried this out quickly and there seems to be a problem with the integration with cartopy. For instance:

fig, ax = pplt.subplots(proj="pcarree")

works as expected, but trying a few different projections such as "robin", "merc" or "npstere" does not and throws an error.

Click to expand trace

---

TypeError Traceback (most recent call last) File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/matplotlib/figure.py:1787, in FigureBase.get_tightbbox(self, renderer, bbox_extra_artists) 1786 try: -> 1787 bbox = ax.get_tightbbox( 1788 renderer, bbox_extra_artists=bbox_extra_artists) 1789 except TypeError:

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/proplot/axes/geo.py:1326, in _CartopyAxes.get_tightbbox(self, renderer, *args, **kwargs) 1324 self._gridliners = [] -> 1326 return super().get_tightbbox(renderer, *args, **kwargs)

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/proplot/axes/base.py:2780, in Axes.get_tightbbox(self, renderer, *args, **kwargs) 2779 self.indicate_inset_zoom() -> 2780 self._tight_bbox = super().get_tightbbox(renderer, *args, **kwargs) 2781 return self._tight_bbox

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/cartopy/mpl/geoaxes.py:499, in GeoAxes.get_tightbbox(self, renderer, *args, **kwargs) 497 self._draw_preprocess(renderer) --> 499 return super().get_tightbbox(renderer, *args, **kwargs)

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/matplotlib/_api/deprecation.py:457, in make_keyword_only..wrapper(*args, **kwargs) 452 warn_deprecated( 453 since, message="Passing the %(name)s %(obj_type)s " 454 "positionally is deprecated since Matplotlib %(since)s; the " 455 "parameter will become keyword-only %(removal)s.", 456 name=name, obj_type=f"parameter of {func.name}()") --> 457 return func(*args, **kwargs)

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/matplotlib/axes/_base.py:4499, in _AxesBase.get_tightbbox(self, renderer, call_axes_locator, bbox_extra_artists, for_layout_only) 4498 for a in bbox_artists: -> 4499 bbox = a.get_tightbbox(renderer) 4500 if (bbox is not None 4501 and 0 < bbox.width < np.inf 4502 and 0 < bbox.height < np.inf):

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/cartopy/mpl/gridliner.py:1243, in Gridliner.get_tightbbox(self, renderer) 1242 self._draw_gridliner(renderer=renderer) -> 1243 bboxes = [c.get_tightbbox(renderer=renderer) 1244 for c in self.get_visible_children()] 1245 if bboxes:

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/cartopy/mpl/gridliner.py:1243, in (.0) 1242 self._draw_gridliner(renderer=renderer) -> 1243 bboxes = [c.get_tightbbox(renderer=renderer) 1244 for c in self.get_visible_children()] 1245 if bboxes:

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/matplotlib/artist.py:365, in Artist.get_tightbbox(self, renderer) 350 """ 351 Like .Artist.get_window_extent, but includes any clipping. 352 (...) 363 Returns None if clipping results in no intersection. 364 """ --> 365 bbox = self.get_window_extent(renderer) 366 if self.get_clip_on():

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/matplotlib/collections.py:311, in Collection.get_window_extent(self, renderer) 308 def get_window_extent(self, renderer=None): 309 # TODO: check to ensure that this does not fail for 310 # cases other than scatter plot legend --> 311 return self.get_datalim(transforms.IdentityTransform())

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/matplotlib/collections.py:270, in Collection.get_datalim(self, transData) 269 if not transform.is_affine: --> 270 paths = [transform.transform_path_non_affine(p) for p in paths] 271 # Don't convert transform to transform.get_affine() here because 272 # we may have transform.contains_branch(transData) but not 273 # transforms.get_affine().contains_branch(transData). But later, 274 # be careful to only apply the affine part that remains.

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/matplotlib/collections.py:270, in (.0) 269 if not transform.is_affine: --> 270 paths = [transform.transform_path_non_affine(p) for p in paths] 271 # Don't convert transform to transform.get_affine() here because 272 # we may have transform.contains_branch(transData) but not 273 # transforms.get_affine().contains_branch(transData). But later, 274 # be careful to only apply the affine part that remains.

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/matplotlib/transforms.py:2445, in CompositeGenericTransform.transform_path_non_affine(self, path) 2444 elif not self._a.is_affine and self._b.is_affine: -> 2445 return self._a.transform_path_non_affine(path) 2446 else:

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/cartopy/mpl/geoaxes.py:175, in InterProjectionTransform.transform_path_non_affine(self, src_path) 174 transformed_geoms = [] --> 175 geoms = cpatch.path_to_geos(src_path) 177 for geom in geoms:

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/cartopy/mpl/patch.py:210, in path_to_geos(path, force_ccw) 208 if geom_collection and all(isinstance(geom, sgeom.LineString) for 209 geom in geom_collection): --> 210 geom_collection = [sgeom.MultiLineString(geom_collection)] 212 # Remove any zero area Polygons

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/shapely/geometry/multilinestring.py:60, in MultiLineString.new(self, lines) 58 return shapely.from_wkt("MULTILINESTRING EMPTY") ---> 60 return shapely.multilinestrings(subs)

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/shapely/decorators.py:77, in multithreading_enabled..wrapped(*args, **kwargs) 76 arr.flags.writeable = False ---> 77 return func(*args, **kwargs) 78 finally:

File ~/miniforge3/envs/ppl_dev/lib/python3.11/site-packages/shapely/creation.py:393, in multilinestrings(geometries, indices, out, **kwargs) 392 if indices is None: --> 393 return lib.create_collection(geometries, typ, out=out, **kwargs) 394 else:

TypeError: ufunc 'create_collection' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

[cvanelteren]

@Holmgren825 I cannot reproduce this error here locally. It also points to a shapely issue rather than proplot. I have cartopy 0.23.0 installed, what version are you running?

For shapely the version is 2.0.5.

[cvanelteren]

The projections in the docs also point to it working fine (see https://proplot--459.org.readthedocs.build/en/459/projections.html). The environment is currently rather unconstrained and would love to add the information in there to prevent a build issue like yours from happening.