helper layers should accept matplotlib palettes

[andy-esch]

matplotlib has a great library of color palettes. It's pretty common to use them for creating charts, etc. A growing pain I've had is getting my maps to have colors aligned with my charts. You can see an example at the bottom of this notebook (https://nbviewer.jupyter.org/github/CartoDB/data-science-book/blob/master/Chapter%202/skater.ipynb). I manually copied and pasted the HTML to get the colors from the legend. It'd be great if instead we could easily pass in a matplotlib color palette to the palette kwarg in any of the color_* functions to help alleviate some of the issues with the workflow.

Here's an example of a standard workflow:

See the full notebook here: https://nbviewer.jupyter.org/github/CartoDB/data-science-book/blob/master/Chapter%201/Visualizing%20spatial%20data%20with%20CARTOframes.ipynb

Notice that we have to do some transformations to get the palette in a format that works with carto vl:

from cartoframes.viz import Map, Layer, basemaps
from cartoframes.viz.helpers import color_continuous_layer, color_bins_layer
from matplotlib import cm

viridis = cm.get_cmap("viridis", 9)

Map(
    color_continuous_layer(
        data_pf,
        "sst",
        palette=[f"rgb({c[0]}, {c[1]}, {c[2]})" for c in 255 * viridis.colors],
        title="Sea Surface Temperature (C)",
        description="ICOADS 1-3 May 2019",
    ),
    basemap=basemaps.darkmatter,
)

I think we should make this easier and just pass in palette=viridis.colors or even palette=viridis. This way it allows people to reuse the palette for any graphs that accompany the visualization.

cc @makella

[makella]

hi @andy-esch! i like this suggestion too thanks for the details and example!

I'm curious about the other way around... for example, is there a way to match CartoColor, ColorBrewer, and also very importantly custom color schemes to the charts that isn't manual?

I would hate for the workflow of using our color schemes and custom color schemes to be too complex to the point where people who have both, only use palettes from mapplotlib... given that the primary purpose of their palettes is for charts, not maps or tested specifically with our basemaps...

I totally agree with supporting a wider variety of defacto Data Science color schemes!! I do wonder though if there is a way to make this really manual process less manual no matter which color scheme you are using to align colors between maps and charts (which I absolutely love the idea of!)

thoughts?

[andy-esch]

Hey @makella good point!

The two workflows I see:

1. User supplies colors to the Layer and then uses the same colors in a plot that accompanies the map

   colors = <colors from matplotlib, custom list, palettable, etc.>
   color_category_layer(df, 'col', palette=colors)
   plot(x, y, palette=colors)
   

2. User wants colors from a layer to be used in a plot that accompanies the map

   color_category_layer(df, 'col', palette='sunset')
   plot(x, y, palette=🤷‍♂)
   

   This feels liked the real problematic workflow because it's not possible (with reasonable effort) to get the color palette from the layer. Jesus has a great proposal on how we could possibly have this information be exposed (https://github.com/CartoDB/cartoframes/issues/792) but the palette information from CARTO VL lies in the js side of things so is sandboxed there to some extent (there is a a workaround). For example, this would be a pretty nice capability:

   layer = color_category_layer(df, 'col', palette='sunset')
   plot(x, y, palette=layer.palette)
   

I did do some work two years ago that added CartoColors to a Python Package: https://jiffyclub.github.io/palettable/

For the near term, we could just have example of how to use palettable with cartoframes to achieve the shared color palettes in the map and the plots. A big problem I see, though, is that the colors in CARTO VL category maps are assigned by mode (most frequent category gets the first color, second most frequent gets the second color, etc.) so the user won't know a priori what the color assignments will be. They could of course specify this up front using the options in the helper, but it is again another barrier to create what should be an easy operation.

[Jesus89]

We are using style helpers instead and are moving to WebSDK. cc @VictorVelarde for awareness.

[VictorVelarde]

Passing a custom palette should be quite simple on any stylecolor helper... just an array of hex colors, and that's it. But I'm not sure if that "integration" should be greater, as suggested here...

I think that we could probably close this one for now, unless you want to add it to the backlog.