Providing an easier API

[almarklein]

In #176 some possibilities for simplifying the API were discussed. This issue is to park that discussion and revisit when the "real" API has settled more.

To summarize

As @Korijn said: One of the things that makes the API in its current state (and as it is in ThreeJS) so accessible/discoverable is that almost everything can be composed just by instantiating the right classes, and providing the right keyword arguments to their constructors. I don't think you need to reach for anything else in almost all examples.

However, I also think it may be overwhelming for some of our users who are not that familiar with the object vs geometry vs material concepts. E.g. scientists who just want to plot something. How are we going to help them?

• Make sure our docs are clear and have a helpful guide to get started.
• We could add API sugar to allow writing less verbose code.
• We could have a higher level API layer on top.

[Korijn]

There is definitely a learning curve, for sure! But it's not very steep. I think the first step should be to step up our documentation game. The API is already super friendly and high quality, and we reduced the boilerplate so much already. I'm very proud of what we (you mostly ;) achieved!

We could have a higher level API layer on top.

Also see #140 and #107. As a first step towards this, we should make sure we have a good understanding of the different use cases, so high level APIs can be tailored for them.

[almarklein]

The direction we're heading: regarding our API we're going for consistency and good documentation instead of adding convenience "shortcuts". An easier (higher level) API would be a separate layer (like vispy 2.0).

[kushalkolar]

Hi, this is something I may be interested in helping to develop and it's stirring a bit of interest in the local neuroscience community over here! The compatibility with the jupyter_rfb is just amazing. I've been looking for something like this for long time, and it seems like the efficiency of the vulkan back-end is really necessary for our visualization needs.

[Korijn]

Hi, this is something I may be interested in helping to develop and it's stirring a bit of interest in the local neuroscience community over here! The compatibility with the jupyter_rfb is just amazing. I've been looking for something like this for long time, and it seems like the efficiency of the vulkan back-end is really necessary for our visualization needs.

Can you explain what type of requirements your community would have? What types of APIs would be convenient for you?

[kushalkolar]

Something like pyqtgraph would probably be great, with flexible callbacks. The way that pygfx can already access variables in its "animate loop" in a non-blocking manner in notebooks is really nice!

I'm trying to visualize 4 movies of neuronal imaging data collected at 11 Hz along with 500 Hz video of a behavioral task that was collected in sync with the neuronal imaging, behavioral-tracks (3D lines) plotted in 3D and PCA plots in 3D (scatter plot in 3D). I use a slider in jupyter notebooks to scroll through timepoints.

Something like this would probably reduce the boilerplate:

# A very matplotlib-like example
from pygfx import high_level_api at hla
from itertools import product

n_rows = 2
n_cols = 2
grid_scenes = hla.GridScenes(2, 2)  # 4x2 gridplot

img_plot = hla.ImagePlot(data=np.random.rand(32, 32).astype(np.float32), vmin=0, vmax=255, cmap='gray')
scatter_plot = hla.ScatterPlot(data=<2D or 3D array>, colors=<array of colors or single color>, alpha=0.5)

# 3D line plots and 2D heatmaps would also be very useful

grid_scenes[0, 0].add_plot(img_plot)
grid_scenes[0, 0].set_interaction(pan=True, zoom=True)

grid_scenes[0, 1].add_plot(scatter_plot)
grid_scenes[0, 1].set_interaction(pan=True, zoom=True, three_d=True)  # 3D interactions


def animate():
    render.render(grid_scenes)
    # callback functions can change attributes if this is in a class (or global variables), and these attributes can be used in the `set()` method
    img_plot.set(data=<new data>, cmap=<new_cmap>)
    scatter_plot.set(data=<new data>...)

[Korijn]

Thanks for sharing your thought process here, it helps us understand better what you're thinking of when reading this issue.

I am immediately reminded of issue #188. Perhaps we should close one of these issues.

There is clearly an expectation from users coming from the python viz and data-science communities to provide a matplotlib-esque API, but we also want to cater to the more traditional graphics & games community. I think if we can do that here with pygfx by stacking abstractions/APIs on top of one another, users will be facilitated well depending on how high-level their needs are:

• Want to develop your own rendering engine? Use wgpu-py.
• Want to develop a plotting library, or a video game, or any graphical application? Use pygfx.
• Want to quickly inspect and visualize datasets? Use <plotting library made with pygfx>. :)

Note that I think that pygfx is also capable of quick inspection and visualization of datasets, but it will take a couple more lines of code to get it done, where a higher level plotting library can provide one-liners such as the ones in your proposal.

The main difference between pygfx and a higher level plotting library is that pygfx tries to be un-opinionated, and allows application developers to build anything they can think of, where a plotting library is more likely to force you into a particular workflow.

I hope I'm not bumming anyone out here, please continue the conversation here if you have any questions or feedback!

[almarklein]

I also think that PyGfx should not offer a plotting API itself, but rather that it'd be a new library that sits on top of PyGfx.

It may be worth noting that I'm working with Cyrille Rossant and Nicolas Rougier on a higher level API. The scope is a bit bigger because the idea is for that API to target multiple backends (at least PyGfx and Datoviz). To that end we're now working on a protocol to serialize "visualization commands". A side-effect of this is that it would support use cases like remote rendering and storing/exporting visualizations. All this is still very experimental. We have a repo where we have some experiments here: https://github.com/vispy/vispy2-sandbox

You indicated that you're interested in helping out. Maybe a good place to start is to start working on a proof of concept for just pygfx. We could than later use that as a basis for an API that uses the serialization step that I mentioned above. I'd recommend having a look at the high level API's of e.g. Bokeh and other tools for inspiration.

[kushalkolar]

Thanks for the ideas! Yup I agree that an independent high level API that just uses pygfx is probably a cleaner solution. I started playing around with making a high level grid plot: https://github.com/kushalkolar/wgpu-viz-prototype/blob/master/gridplot.py

usage is quite simple:

canvas = WgpuCanvas()
renderer = gfx.renderers.WgpuRenderer(canvas)

grid_plot = GridPlot(
    canvas=canvas,
    renderer=renderer,
    grid_shape=(2, 3),  # can create an [m x n] grid
    cameras=np.array([['o', 'o', 'o'], ['o', 'o', 'o']]),  # just orthographic cams for now
    controllers=np.array([[0, 3, 1], [2, 2, 3]])  # each number is a unique pan-zoom controller
)

for subplot in grid_plot:
    img = (np.random.rand(512, 512) * 255).astype(np.float32)
    
    image_graphic = Image(data=img, vmin=0, vmax=255)
    subplot.add_graphic(image_graphic)
    
canvas.request_draw(grid_plot.animate)
canvas

https://user-images.githubusercontent.com/9403332/165017809-b41ed157-9be7-47d2-9854-0290fd8233b9.mp4

I might play around with it a bit more over the next 2 weeks and I am on vacation for a while after that, so I'll probably get more into it in mid-May/June!

[kushalkolar]

Started an experimental package for a high level API if you're interested in checking it out, open to ideas! It's in very very early stages.

Usage examples: https://github.com/kushalkolar/fastplotlib/tree/master/examples

[almarklein]

fastplotlib 😆

Will check it out, very interesting!

[almarklein]

I think this can be closed:

• We accept that our API is somewhat verbose.
• We try to make things simpler by using good defaults.
• We have show() for quick stuff.
• The rest is up to fastplotlib at al :)