dash_table cannot handle pandas.NA

[kasuteru]

Pandas now uses an experimental NA declarator by default: pandas.NA of type pandas._libs.missing.NAType. More details here.

Currently, dash_table throws an "Error loading layout" during runtime whenever it encounters a table with one of these values. This is very hard to debug since a) it might not occur in development / testing and crash the app in production b) the error message is very unspecific, making it hard to find the problem.

Example code:

import dash
import dash_table
import dash_html_components as html
import pandas as pd


# Using this dataframe works fine:
df = pd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})

# Using this dataframe crashes the app:
df = pd.DataFrame({'col1': [1, 2], 'col2': [3, pd.NA]})

app = dash.Dash(__name__)

app.layout = html.Div([
    dash_table.DataTable(
        id='datatable',
        columns=[
            {"name": i, "id": i} for i in df.columns
        ],
        data=df.to_dict('records')
    )
])


if __name__ == '__main__':
    app.run_server(debug=True)

[PolycarpeLeGrand]

I just spent two hours trying to debug a callback updating a table's columns and data, turns out this was the problem. I kept getting a dash.exceptions.InvalidCallbackReturnValue when updating with some specific data, the error message saying the callback returned a tuple. This left me pretty confused since my callback always returns df.to_dict('records'), and i couldn't find any difference in the type or format of the data explaining why it would crash in some cases. Turns out there were some <NA> values sprinkled around my data causing the error, removing them seems to have fixed the issue.

Anyways, thanks for pointing this out, i'm not sure i would have firugred it out. Here's the full return statement and the error if it's any help:

return [{'name': map_name(c), 'id': c} for c in cols], df[cols].to_dict('records')

dash.exceptions.InvalidCallbackReturnValue: The callback for `[<Output `fmsq-table.columns`>, <Output `fmsq-table.data`>]`
returned a value having type `tuple`
which is not JSON serializable.