Incorrect handling of np.datetime64 raises DTypePromotionError

[yt87]

Software versions

Python version : 3.12.9 | packaged by conda-forge | (main, Feb 14 2025, 08:00:06) [GCC 13.3.0] IPython version : 8.32.0 Tornado version : 6.4.2 NumPy version : 2.1.3 Bokeh version : 3.6.3 BokehJS static path : /home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/bokeh/server/static node.js version : v23.7.0 ERROR: [Errno 2] No such file or directory: PosixPath('npm')

Browser name and version

No response

Jupyter notebook / Jupyter Lab version

No response

Expected behavior

I raised the issue on hvplot github: https://github.com/holoviz/hvplot/issues/1500. Later I found that this might be an exclusively Bokeh problem, so I cross-reference it here. For completeness I enclose console stack trace.

My solution: call convert_datetime_array in ColumnDataSource.__init__:

    def __init__(self, *args: TAny, **kwargs: TAny) -> None:
        ''' If called with a single argument that is a dict or
        ``pandas.DataFrame``, treat that implicitly as the "data" attribute.

        '''
        if len(args) == 1 and "data" not in kwargs:
            kwargs["data"] = args[0]

        # TODO (bev) invalid to pass args and "data", check and raise exception
        raw_data: DataDict = kwargs.pop("data", {})

        import pandas as pd
        if not isinstance(raw_data, dict):
            if isinstance(raw_data, pd.DataFrame):
                raw_data = self._data_from_df(raw_data)
            elif isinstance(raw_data, pd.core.groupby.GroupBy):
                raw_data = self._data_from_groupby(raw_data)
            else:
                raise ValueError(f"expected a dict or pandas.DataFrame, got {raw_data}")
        super().__init__(**kwargs)
        self.data.update(raw_data)
        # GT fix
        for key, values in self.data.items():
            # Apply the transformation if the data contains datetimes
            if isinstance(values, np.ndarray) and values.dtype.kind.lower() == 'm':
                self.data[key] = convert_datetime_array(values)

Observed behavior

The plot is displayed, however an unending list of exception tracebacks is printed.

Example code

import hvplot.streamz  # noqa
from streamz.dataframe import Random
df = Random(interval='5s', freq='1s')
df.hvplot()

Stack traceback or browser console output

jlab_core.3e79afb39b…9afb39b563f309a5d:1 TypeError: Cannot read properties of undefined (reading 'isRunning')
    at jlab_core.3e79afb39b…63f309a5d:1:1250545
    at Array.forEach (<anonymous>)
    at ze.updateRunningStatus (jlab_core.3e79afb39b…63f309a5d:1:1250440)
    at ze.onExecutionScheduled (jlab_core.3e79afb39b…63f309a5d:1:1250020)
    at m (jlab_core.3e79afb39b…63f309a5d:1:1831671)
    at Object.l [as emit] (jlab_core.3e79afb39b…63f309a5d:1:1831347)
    at a.emit (jlab_core.3e79afb39b…63f309a5d:1:1829184)
    at onCellExecutionScheduled (jlab_core.3e79afb39b…63f309a5d:1:1148687)
    at Object.u [as runCell] (jlab_core.3e79afb39b…63f309a5d:1:1125724)
    at c (jlab_core.3e79afb39b…63f309a5d:1:1148789)
jlab_core.3e79afb39b…9afb39b563f309a5d:1 Failed to update the table of contents. TypeError: Cannot read 
properties of undefined (reading 'isRunning')
    at jlab_core.3e79afb39b…63f309a5d:1:1250545
    at Array.forEach (<anonymous>)
    at ze.updateRunningStatus (jlab_core.3e79afb39b…63f309a5d:1:1250440)
    at ze.getHeadings (jlab_core.3e79afb39b…63f309a5d:1:1248614)
    at ze.refresh (jlab_core.3e79afb39b…63f309a5d:1:1589945)
    at s (jlab_core.3e79afb39b…63f309a5d:1:1587862)
    at m (jlab_core.3e79afb39b…63f309a5d:1:1831671)
    at Object.l [as emit] (jlab_core.3e79afb39b…63f309a5d:1:1831347)
    at a.emit (jlab_core.3e79afb39b…63f309a5d:1:1829184)
    at jlab_core.3e79afb39b…563f309a5d:1:475509
bokeh-3.6.3.min.js:185 [bokeh 3.6.3] setting log level to: 'info'
jlab_core.3e79afb39b…9afb39b563f309a5d:1 TypeError: Cannot set properties of undefined (setting 'isRunning')
    at jlab_core.3e79afb39b…63f309a5d:1:1249667
    at Array.forEach (<anonymous>)
    at ze.onExecuted (jlab_core.3e79afb39b…63f309a5d:1:1249425)
    at m (jlab_core.3e79afb39b…63f309a5d:1:1831671)
    at Object.l [as emit] (jlab_core.3e79afb39b…63f309a5d:1:1831347)
    at a.emit (jlab_core.3e79afb39b…63f309a5d:1:1829184)
    at onCellExecuted (jlab_core.3e79afb39b…63f309a5d:1:1148612)
    at Object.u [as runCell] (jlab_core.3e79afb39b…63f309a5d:1:1126167)
    at async Promise.all (index 0)
    at async execute (jlab_core.3e79afb39b…b563f309a5d:1:72115)
panel.min.js:157 Python failed with the following traceback: 
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/pyviz_comms/__init__.py _handle_msg L339
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/panel/viewable.py _on_msg L493
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/bokeh/protocol/messages/patch_doc.py apply_to_document L104
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/bokeh/document/callbacks.py invoke_with_curdoc L453
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/bokeh/protocol/messages/patch_doc.py <lambda> L104
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/bokeh/document/document.py apply_json_patch L391
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/bokeh/document/events.py handle_event L244
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/bokeh/document/events.py _handle_event L566
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/bokeh/models/sources.py _stream L582
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/bokeh/core/property/wrappers.py _stream L492
/home/george/miniforge3/envs/aawu/lib/python3.12/site-packages/numpy/lib/_function_base_impl.py append L5692
	DTypePromotionError: The DType <class 'numpy.dtypes.DateTime64DType'> could not be promoted by <class 'numpy.dtypes.Float64DType'>. This means that no common DType exists for the given inputs. For example they cannot be stored in a single array unless the dtype is `object`. The full list of DTypes is: (<class 'numpy.dtypes.DateTime64DType'>, <class 'numpy.dtypes.Float64DType'>)

Screenshots

No response

[bryevdv]

@yt87 Are you able to provide a pure-Bokeh MRE? Even if there is a simple fix, it will need to be accompanied with tests and those should be only Bokeh code. I am not sure how to reproduce this based on the information above. cc @philippjfr

[yt87]

This issue occurs only in jupyter. When I run a slightly modified notebook

import hvplot.streamz  # noqa
from streamz.dataframe import Random
import panel as pn
df = Random(interval='5s', freq='1s')
plt = df.hvplot()
pane = pn.panel(plt)
pane

as a panel app, the output is as expected. I tried to find out what is going on by adding some print statements inside bokeh package. The console traceback above shows that the appended dataframe index values are already converted to floats before np.append is called. When the same code runs as a standalone app, the conversion does not occur yet at that point, both arrays have the same type np.datetime64. The conversion from us (int64) to ms (float64) happens later.

I am at a loss here.

[bryevdv]

@yt87 I'm not sure what to add without a pure-Bokeh reproducer there is not much we can do. I am inclined to close with norepro until/unless a pure Bokeh MRE can be found, then we could certainly re-open.

[mosc9575]

The problem is related to Numpy 2.0 and was first documented in https://github.com/bokeh/bokeh/issues/14004.

For me it looks like the source inside the Random class is initialized with the wrong datatype and causes the reported error.