Optimizing Python <-> JavaScript communication for streaming data

[martinRenou]

cc. @kaiayoung @mbektasbbg

When using ipywidgets, it is not supported to change a subset of the data. e.g.:

my_data_grid.data[26][32] = 3.4

will not send the proper message to the JavaScript side for it to update the grid view. For this to work, it is needed to assign the entire data again. Which means doing:

old_data = my_data_grid.data
old_data[26][32] = 3.4
# Making a copy of the data so that ipywidgets sees the change...
my_data_grid.data = old_data[:]

This might affect performances drastically for streaming data on big grids...

But there is a way around this. If we provide means to update the data using custom comm messages, we could have very decent performances. Which means doing something like:

class DataGrid(DOMWidget):
    [...]

    def update_cell(self, row, column, value):
        self.send({'action': 'update_cell', args: [row, column, value]})
    
    def update_row(self, row, value):
        self.send({'action': 'update_row', args: [row, value]})
    
    def update_column(self, column, value):
        self.send({'action': 'update_column', args: [column, value]})

and on the JavaScript side (this is pseudo code, not tested):

this.model.on('msg:custom', (message) => {
  if (message.action === 'update_cell') {
    const [row, column, value] = message.args;
    this.model.data[row, column] = value
  }

  [...]
});

[martinRenou]

Thought: we could even implement the __setitem__ method on the DataGrid that would call one of those methods. Adding support for slicing etc.

[wolfv]

I think this should maybe happen as a prototype in the datagrid, but the appropriate place for something like this to live would be in widgets itself, no?

[wolfv]

(sry, didn't mean to close) :)

[martinRenou]

I totally agree

[martinRenou]

One issue I see with this approach could be linking the data with another widget. I am not sure it would work. But we might be able to find a workaround for that.