Update imports and func references to support up to latest matplotlib (3.8.3)

[riley-brady]

• Adds support up to matplotlib==3.6.0 by leveraging _gen_cmap_registry() to set up colormap database.
• Adds support up to latest matplotlib==3.8.3 by referencing hidden _fontconfig_pattern module for setting up fontconfig parser.

[chudlerk]

I get this error when I call pplt.figure() :(

AttributeError: 'Figure' object has no attribute '_cachedRenderer'

Matplotlib v3.8.3 Python 3.11.8

[riley-brady]

Hi Kyle @chudlerk! Thanks for pointing this out. I was just trying to get the package to import but clearly didn't do any testing. It's hard to capture the depth of dependency issues here until https://github.com/proplot-dev/proplot/pull/413 is merged.

Might need to defer to @lukelbd on this, since he has a better idea of what this _cachedRenderer does. It looks like it was dropped starting with 3.6.0. See https://matplotlib.org/stable/api/prev_api_changes/api_changes_3.6.0.html#renderer-optional-for-get-tightbbox-and-get-window-extent.

They do have a Figure._get_renderer() that seems to have replaced the cached renderer.

There's two spots in proplot this needs to be addressed.

1. Under figure.py, Figure._get_renderer(). That just copies the call from matplotlib's tight_layout, which seemingly was deprecated at 3.6. This method might be able to check matplotlib version and say <3.6 use that block, otherwise just run Figure._get_renderer() from matplotlib.Figure.

2. Under figure.py, Figure._context_adjusting():

    def _context_adjusting(self, cache=True):
        """
        Prevent re-running auto layout steps due to draws triggered by figure
        resizes. Otherwise can get infinite loops.
        """
        kw = {'_is_adjusting': True}
        if not cache:
            kw['_cachedRenderer'] = None  # temporarily ignore it
        return context._state_context(self, **kw)

I'm less sure of what's going on here and how this needs to be modified for more recent versions.

[riley-brady]

@lukelbd , I gave it a little bit of a try today and can't really figure out what the renderer stuff is doing. I have it turned on so you can push commits directly to this if you'd like to try this weekend.

[riley-brady]

@chudlerk I've found success on my work stack just pinning matplotlib==3.5.3 and that will jive with all the most recent xarray/xclim/dask/metpy, etc. etc. If you do these small import fixes from this PR with that I think it's a good temporary solution.

[chudlerk]

Thanks Riley @riley-brady! I've got a working environment also by pinning matplotlib that way. At one point I think I had some issues with newer versions of python not working with the old matplotlib version. But everything is working fine with 3.9.

[riley-brady]

I was getting some weird errors with:

python==3.10.13
matplotlib==3.4.3
proplot==0.9.7
cartopy=0.22

Where cartopy was throwing an error related to packaging.version. It only seemed to resolve by bumping matplotlib to 3.5.3. Was able to keep python at 3.10.13 this way. Not sure how that matplotlib version impacted it, because the packaging version and python version was fixed.

[reemagit]

Hello, I had the same issue with cmap_d and locally modified the code in a similar way as shown above. It imports fine, but the I also got the _cachedRenderer error. So I modified the line in proplot'sfigure.py:

if self._cachedRenderer:
            renderer = self._cachedRenderer 

to

if hasattr(self,'_cachedRenderer') and self._cachedRenderer:
            renderer = self._cachedRenderer 

I was testing some example code:

fig,axs = pplt.subplots(nrows=1,ncols=1)
axs.scatter([0,2,3,4],[1,2,4,2])

but now I get a new error:

ValueError: 'Fire' is not a valid value for cmap; supported values are 'Accent', 'Accent_r', 'Blues', 'Blues_r', 'BrBG', 'BrBG_r', 'BuGn', 'BuGn_r', 'BuPu', 'BuPu_r', 'CMRmap', 'CMRmap_r', 'Dark2', 'Dark2_r', 'GnBu', 'GnBu_r', 'Greens', 'Greens_r', 'Greys', 'Greys_r', 'OrRd', 'OrRd_r', 'Oranges', 'Oranges_r', 'PRGn', 'PRGn_r', 'Paired', 'Paired_r', 'Pastel1', 'Pastel1_r', 'Pastel2', 'Pastel2_r', 'PiYG', 'PiYG_r', 'PuBu', 'PuBuGn', 'PuBuGn_r', 'PuBu_r', 'PuOr', 'PuOr_r', 'PuRd', 'PuRd_r', 'Purples', 'Purples_r', 'RdBu', 'RdBu_r', 'RdGy', 'RdGy_r', 'RdPu', 'RdPu_r', 'RdYlBu', 'RdYlBu_r', 'RdYlGn', 'RdYlGn_r', 'Reds', 'Reds_r', 'Set1', 'Set1_r', 'Set2', 'Set2_r', 'Set3', 'Set3_r', 'Spectral', 'Spectral_r', 'Wistia', 'Wistia_r', 'YlGn', 'YlGnBu', 'YlGnBu_r', 'YlGn_r', 'YlOrBr', 'YlOrBr_r', 'YlOrRd', 'YlOrRd_r', 'afmhot', 'afmhot_r', 'autumn', 'autumn_r', 'binary', 'binary_r', 'bone', 'bone_r', 'brg', 'brg_r', 'bwr', 'bwr_r', 'cividis', 'cividis_r', 'cool', 'cool_r', 'coolwarm', 'coolwarm_r', 'copper', 'copper_r', 'cubehelix', 'cubehelix_r', 'flag', 'flag_r', 'gist_earth', 'gist_earth_r', 'gist_gray', 'gist_gray_r', 'gist_heat', 'gist_heat_r', 'gist_ncar', 'gist_ncar_r', 'gist_rainbow', 'gist_rainbow_r', 'gist_stern', 'gist_stern_r', 'gist_yarg', 'gist_yarg_r', 'gnuplot', 'gnuplot2', 'gnuplot2_r', 'gnuplot_r', 'gray', 'gray_r', 'hot', 'hot_r', 'hsv', 'hsv_r', 'inferno', 'inferno_r', 'jet', 'jet_r', 'magma', 'magma_r', 'nipy_spectral', 'nipy_spectral_r', 'ocean', 'ocean_r', 'pink', 'pink_r', 'plasma', 'plasma_r', 'prism', 'prism_r', 'rainbow', 'rainbow_r', 'seismic', 'seismic_r', 'spring', 'spring_r', 'summer', 'summer_r', 'tab10', 'tab10_r', 'tab20', 'tab20_r', 'tab20b', 'tab20b_r', 'tab20c', 'tab20c_r', 'terrain', 'terrain_r', 'turbo', 'turbo_r', 'twilight', 'twilight_r', 'twilight_shifted', 'twilight_shifted_r', 'viridis', 'viridis_r', 'winter', 'winter_r'

Unfortunately I can't use matplotlib==3.5.3 since I have another package that requires matplotlib>=3.6.0.

Has anyone found a workaround for this?

[riley-brady]

@reemagit , in the meantime you can just use a different colormap from that list. It looks like Fire is the default and not being registered. Just pass in cmap='viridis' to scatter.

In the meantime:

1. Check that your packaged source code has the cmaps folder with Fire.json in it. (My pathing is e.g. ~/miniforge3/envs/<conda_env_name>/lib/python3.10/site-packages/proplot). If there is a cmaps folder with Fire.json in it, that means that with the modern matplotlib version it's just not registering the custom ones correctly.
2. If (1) is true, it probably has to do with my changes to proplot/colors.py. See the changes in this PR. If you haven't implemented those -- try them out. You just need to make sure that database = mcm._gen_cmap_registry() creates the proper colormap registry with Fire.json included.

[reemagit]

Thanks for the suggestions. With the PR changes and using the standard colormaps it seems to work.

I tried understanding why it wouldn't load the colormaps. Posting what I understood below in case it can be useful:

I checked and the 'Fire.json' file exists, and proplot is able to load it. However it doesn't get loaded in the global ColormapRegistry so it throws an error. I think this is because the line in matplotlib mpl._api.check_in_list(sorted(mpl.cm._colormaps) checks if Fire exists in mpl.cm._colormaps, but in the corrected code we add the colormap database to setattr(mcm, attr, database), with attr being either "_cmap_registry" or "cmap_d". So at the end of the proplot's _init_cmap_database function I changed it to

#setattr(mcm, attr, database) # Line that I removed
setattr(mcm, '_colormaps', database)
setattr(mpl, 'colormaps', mcm._colormaps)

In this way the Database is loaded in the global mpl.colormaps and 'mcm._colormaps' objects.

Even after doing this, it still wouldn't find the 'Fire' colormap. Then I saw that the matplotlib.colormaps contained the 'fire' colormap (lowercase). As I found out, the ColormapDatabase is not case-sensitive when requesting a colormap, so it works regardless. However, the function checking if 'Fire' exists uses

_api.check_in_list(sorted(_colormaps), name=name)

The sorted functions converts the keys of _colormaps to a list, and so it becomes case-sensitive. So the final solution was to just call the "fire" (lowercase) colormap, and it would work. So the final solution if one wants to use the proplot's colormaps is to change the lines above and always specify the lowercase name.

Not sure if my workaround introduced other issues, but it seems to be working so far. Hope this helps.

[kastnerp]

Works for me on Python 3.11.5 (main, Sep 11 2023, 08:31:25) [Clang 14.0.6 ] on darwin and matplotlib 3.8.3 matplotlib-inline 0.1.6!

[riley-brady]

Works for me on Python 3.11.5 (main, Sep 11 2023, 08:31:25) [Clang 14.0.6 ] on darwin and matplotlib 3.8.3 matplotlib-inline 0.1.6!

Is this with @reemagit's fixes? Have you tried plotting some stuff? PR head will only fix imports, but not plotting.

[kastnerp]

I installed from the commit c0bf6b6b8ec047e302eea78631942467575e75cb and plotting works without an issue on my end.

[austin-hoover]

matplotlib 3.8.4 proplot 1.0.4

I tried

import matplotlib.pyplot as plt
import numpy as np
import proplot as pplt

x = y = np.linspace(-1.0, 1.0, 50)
values = np.random.rand(x.size, y.size)

fig, ax = pplt.subplots()
ax.pcolormesh(x, y, values.T, cmap="fire")
plt.show()

and got the following error:

Traceback (most recent call last):
  File "/Users/46h/repo/proplot-test/test/test_plot_2d.py", line 9, in <module>
    fig, ax = pplt.subplots(ncols=2, nrows=2)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/46h/repo/proplot-test/proplot/ui.py", line 232, in subplots
    axs = fig.add_subplots(*args, rc_kw=rc_kw, **kwsubs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/46h/repo/proplot-test/proplot/figure.py", line 1407, in add_subplots
    return self._add_subplots(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/46h/repo/proplot-test/proplot/figure.py", line 1170, in _add_subplots
    axs[idx] = self.add_subplot(ss, **kw)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/46h/repo/proplot-test/proplot/figure.py", line 1393, in add_subplot
    return self._add_subplot(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/46h/repo/proplot-test/proplot/figure.py", line 1063, in _add_subplot
    ax = super().add_subplot(ss, _subplot_spec=ss, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/46h/miniconda3/envs/proplot-test/lib/python3.11/site-packages/matplotlib/figure.py", line 782, in add_subplot
    ax = projection_class(self, *args, **pkw)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/46h/repo/proplot-test/proplot/axes/cartesian.py", line 343, in __init__
    super().__init__(*args, **kwargs)
  File "/Users/46h/repo/proplot-test/proplot/axes/plot.py", line 1279, in __init__
    super().__init__(*args, **kwargs)
  File "/Users/46h/repo/proplot-test/proplot/axes/base.py", line 828, in __init__
    self._apply_auto_share()
  File "/Users/46h/repo/proplot-test/proplot/axes/base.py", line 1397, in _apply_auto_share
    child._sharey_setup(parent)
  File "/Users/46h/repo/proplot-test/proplot/axes/cartesian.py", line 640, in _sharey_setup
    self._sharey_limits(sharey)
  File "/Users/46h/repo/proplot-test/proplot/axes/cartesian.py", line 583, in _sharey_limits
    self.get_shared_y_axes().join(self, sharey)  # share limit/scale changes
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'GrouperView' object has no attribute 'join'. Did you mean: 'joined'?

I updated this line to self.sharey(sharey) and it worked. I don't have any issue with colormaps.

[austin-hoover]

Running

fig, ax = pplt.subplots()

in JupyterLab throws the cached renderer error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File [~/miniconda3/envs/proplot-test/lib/python3.11/site-packages/matplotlib/pyplot.py:197](http://localhost:8888/~/miniconda3/envs/proplot-test/lib/python3.11/site-packages/matplotlib/pyplot.py#line=196), in _draw_all_if_interactive()
    195 def _draw_all_if_interactive() -> None:
    196     if matplotlib.is_interactive():
--> 197         draw_all()

File [~/miniconda3/envs/proplot-test/lib/python3.11/site-packages/matplotlib/_pylab_helpers.py:132](http://localhost:8888/~/miniconda3/envs/proplot-test/lib/python3.11/site-packages/matplotlib/_pylab_helpers.py#line=131), in Gcf.draw_all(cls, force)
    130 for manager in cls.get_all_fig_managers():
    131     if force or manager.canvas.figure.stale:
--> 132         manager.canvas.draw_idle()

File [~/miniconda3/envs/proplot-test/lib/python3.11/site-packages/matplotlib/backend_bases.py:1893](http://localhost:8888/~/miniconda3/envs/proplot-test/lib/python3.11/site-packages/matplotlib/backend_bases.py#line=1892), in FigureCanvasBase.draw_idle(self, *args, **kwargs)
   1891 if not self._is_idle_drawing:
   1892     with self._idle_draw_cntx():
-> 1893         self.draw(*args, **kwargs)

File [~/repo/proplot-test/proplot/figure.py:463](http://localhost:8888/~/repo/proplot-test/proplot/figure.py#line=462), in _add_canvas_preprocessor.<locals>._canvas_preprocess(self, *args, **kwargs)
    461 ctx3 = rc.context(fig._render_context)  # draw with figure-specific setting
    462 with ctx1, ctx2, ctx3:
--> 463     fig.auto_layout()
    464     return func(self, *args, **kwargs)

File [~/repo/proplot-test/proplot/figure.py:1443](http://localhost:8888/~/repo/proplot-test/proplot/figure.py#line=1442), in Figure.auto_layout(self, renderer, aspect, tight, resize)
   1440 # *Impossible* to get notebook backend to work with auto resizing so we
   1441 # just do the tight layout adjustments and skip resizing.
   1442 gs = self.gridspec
-> 1443 renderer = self._get_renderer()
   1444 if aspect is None:
   1445     aspect = True

File [~/repo/proplot-test/proplot/figure.py:891](http://localhost:8888/~/repo/proplot-test/proplot/figure.py#line=890), in Figure._get_renderer(self)
    887 def _get_renderer(self):
    888     """
    889     Get a renderer at all costs. See matplotlib's tight_layout.py.
    890     """
--> 891     if self._cachedRenderer:
    892         renderer = self._cachedRenderer
    893     else:

AttributeError: 'Figure' object has no attribute '_cachedRenderer'

But running

fig, ax = pplt.subplots()
plt.show()

works just fine.

It seems to work if I delete the if statement above.

[austin-hoover]

There still seems to be a cmap registry issue but only for scatter plots.

x = np.random.uniform(size=(500, 2))

fig, ax = pplt.subplots()
ax.scatter(x[:, 0], x[:, 1])

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[15], line 4
      1 x = np.random.uniform(size=(500, 2))
      3 fig, ax = pplt.subplots()
----> 4 ax.scatter(x[:, 0], x[:, 1])

File [~/repo/proplot-test/proplot/internals/inputs.py:296](http://localhost:8888/~/repo/proplot-test/proplot/internals/inputs.py#line=295), in _preprocess_or_redirect.<locals>._decorator.<locals>._preprocess_or_redirect(self, *args, **kwargs)
    293             ureg.setup_matplotlib(True)
    295 # Call main function
--> 296 return func(self, *args, **kwargs)

File [~/repo/proplot-test/proplot/axes/plot.py:3271](http://localhost:8888/~/repo/proplot-test/proplot/axes/plot.py#line=3270), in PlotAxes.scatter(self, *args, **kwargs)
   3267 """
   3268 %(plot.scatter)s
   3269 """
   3270 kwargs = _parse_vert(default_vert=True, **kwargs)
-> 3271 return self._apply_scatter(*args, **kwargs)

File [~/repo/proplot-test/proplot/axes/plot.py:3239](http://localhost:8888/~/repo/proplot-test/proplot/axes/plot.py#line=3238), in PlotAxes._apply_scatter(self, xs, ys, ss, cc, vert, **kwargs)
   3237 for _, n, x, y, s, c, kw in self._iter_arg_cols(xs, ys, ss, cc, **kw):
   3238     kw['s'], kw['c'] = s, c  # make _parse_cycle() detect these
-> 3239     kw = self._parse_cycle(n, cycle_manually=cycle_manually, **kw)
   3240     *eb, kw = self._add_error_bars(x, y, vert=vert, default_barstds=True, **kw)
   3241     *es, kw = self._add_error_shading(x, y, vert=vert, color_key='c', **kw)

File [~/repo/proplot-test/proplot/axes/plot.py:2346](http://localhost:8888/~/repo/proplot-test/proplot/axes/plot.py#line=2345), in PlotAxes._parse_cycle(self, ncycle, cycle, cycle_kw, cycle_manually, return_cycle, **kwargs)
   2344         props[prop] = key
   2345 if props:
-> 2346     dict_ = next(parser.prop_cycler)
   2347     for prop, key in props.items():
   2348         value = dict_[prop]

AttributeError: '_process_plot_var_args' object has no attribute 'prop_cycler'

fig, ax = pplt.subplots()
ax.scatter(x[:, 0], x[:, 1], c="black")

ValueError: 'fire' is not a valid value for cmap

fig, ax = pplt.subplots()
ax.scatter(x[:, 0], x[:, 1], c=np.linspace(0.0, 1.0, x.shape[0]), cmap="viridis")

ValueError: 'fire' is not a valid value for cmap

It works if you change the rc param:

pplt.rc["cmap.sequential"] = "viridis"

fig, ax = pplt.subplots()
ax.scatter(x[:, 0], x[:, 1], c=np.linspace(0.0, 1.0, x.shape[0]), cmap="fire")