Streaming new data to tabulator widget with show_index=False raises a ValueError exception

[sissh]

When streaming new data to a tabulator widget, if the show_index value is set to False then a ValueError exception is raised. It happens with both DataFrames and Dictionaries.

This code below demonstrates it :

import pandas as pd
import panel as pn

df = pd.DataFrame(np.random.randn(10, 5), columns=list('ABCDE'))
tabulator=pn.widgets.Tabulator(df, show_index=False)
def stream_data(event):
    stream_dic = {
        "A":0,
        "B":0,
        "C":0,
        "D":0,
        "E":0
    }
    stream_df = pd.DataFrame(np.random.randn(10, 5), columns=list('ABCDE'))
    tabulator.stream(stream_df)

button = pn.widgets.Button(name="stream")
button.on_click(stream_data)
pn.Row(tabulator, button)

If you set show_index to True, the new data is streamed as expected. This did not happen in version 0.12.6.

Current environment : Panel 0.13.0 Pandas 1.3.2 Jupyter Lab

[hoxbro]

The traceback

Traceback (most recent call last):
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/pyviz_comms/__init__.py", line 338, in _handle_msg
    self._on_msg(msg)
  File "/home/shh/Development/holoviz/panel/panel/viewable.py", line 273, in _on_msg
    patch.apply_to_document(doc, comm.id)
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/protocol/messages/patch_doc.py", line 115, in apply_to_document
    invoke_with_curdoc(doc, lambda: doc.apply_json_patch(self.content, setter))
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/document/callbacks.py", line 408, in invoke_with_curdoc
    return f()
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/protocol/messages/patch_doc.py", line 115, in <lambda>
    invoke_with_curdoc(doc, lambda: doc.apply_json_patch(self.content, setter))
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/document/document.py", line 391, in apply_json_patch
    DocumentPatchedEvent.handle_json(self, event_json, references, setter)
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/document/events.py", line 259, in handle_json
    handler(doc, event_json, references, setter)
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/document/events.py", line 300, in _handle_json
    cb(event_json["msg_data"])
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/document/callbacks.py", line 355, in trigger_json_event
    model._trigger_event(event)
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/util/callback_manager.py", line 119, in _trigger_event
    self.document.callbacks.notify_event(cast(Model, self), event, invoke)
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/document/callbacks.py", line 247, in notify_event
    invoke_with_curdoc(doc, callback_invoker)
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/document/callbacks.py", line 408, in invoke_with_curdoc
    return f()
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/util/callback_manager.py", line 115, in invoke
    cast(EventCallbackWithEvent, callback)(event)
  File "/home/shh/Development/holoviz/panel/panel/reactive.py", line 356, in _comm_event
    self._process_bokeh_event(event)
  File "/home/shh/Development/holoviz/panel/panel/reactive.py", line 306, in _process_bokeh_event
    self._process_event(event)
  File "/home/shh/Development/holoviz/panel/panel/widgets/button.py", line 136, in _process_event
    self.clicks += 1
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 358, in _f
    instance_param.__set__(obj, val)
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 360, in _f
    return f(self, obj, val)
  File "/home/shh/Development/holoviz/param/param/__init__.py", line 621, in __set__
    super(Dynamic,self).__set__(obj,val)
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 360, in _f
    return f(self, obj, val)
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 1249, in __set__
    obj.param._call_watcher(watcher, event)
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 2041, in _call_watcher
    self_._execute_watcher(watcher, (event,))
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 2023, in _execute_watcher
    watcher.fn(*args, **kwargs)
  File "/tmp/ipykernel_11034/3143372746.py", line 16, in stream_data
    tabulator.stream(stream_df)
  File "/home/shh/Development/holoviz/panel/panel/widgets/tables.py", line 1253, in stream
    super().stream(stream_value, rollover, reset_index)
  File "/home/shh/Development/holoviz/panel/panel/widgets/tables.py", line 571, in stream
    self._stream(stream_value, rollover)
  File "/home/shh/Development/holoviz/panel/panel/util.py", line 379, in wrapped
    fn(self, *args, **kwargs)
  File "/home/shh/Development/holoviz/panel/panel/widgets/tables.py", line 1242, in _stream
    super()._stream(stream, rollover)
  File "/home/shh/Development/holoviz/panel/panel/util.py", line 379, in wrapped
    fn(self, *args, **kwargs)
  File "/home/shh/Development/holoviz/panel/panel/reactive.py", line 708, in _stream
    m.source.stream(stream, rollover)
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/models/sources.py", line 450, in stream
    self._stream(new_data, rollover)
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/bokeh/models/sources.py", line 532, in _stream
    raise ValueError("Must stream updates to all existing columns (extra: %s)" % ", ".join(sorted(extra)))
ValueError: Must stream updates to all existing columns (extra: index)

[sissh]

Circling back on that issue, I've found out that simply adding "index" in the "hidden_columns" parameter somewhat fixes this issue. It's a bit counter intuitive because you're explicitly showing the index only to hide it but it works.

tabulator=pn.widgets.Tabulator(df, show_index=True, hidden_columns=["index"])