[BUG] pygwalker widget does not render in databricks

[ewagner70]

Describe the bug pygwalker widget doose not get rendered in databricks. Only

Loading the widget is using longer than expected. We suggest the following ...

To Reproduce Steps to reproduce the behavior: open databricks noteook and insert the following code:

import pandas as pd
import pygwalker as pyg
iris = pd.read_csv('https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv')
pyg.walk(iris)

Expected behavior pyg.walk gets rendered as expected.

Screenshots here is the console output

Versions

• pygwalker version: 0.4.9.3
• databricks version: 14.3 LTS
• python version: 3.10.12
• browser: Edge, Chrome, doesn't matter ...

Additional context

[longxiaofei]

Hi @ewagner70 , thanks for your feedback.

I will try to reproduce this bug in databricks and find out the cause.

[longxiaofei]

The size of pygwalker's HTML is around 19MB, which seems to have triggered the limitation of Databricks ipywidgets:

https://docs.databricks.com/en/notebooks/notebook-limitations.html#ipywidgets

The maximum message payload size for an ipywidget is 5 MB. Widgets that use images or large text data may not be properly rendered.

I am looking for a solution, and this may take some time.

[ewagner70]

The size of pygwalker's HTML is around 19MB, which seems to have triggered the limitation of Databricks ipywidgets:

https://docs.databricks.com/en/notebooks/notebook-limitations.html#ipywidgets

The maximum message payload size for an ipywidget is 5 MB. Widgets that use images or large text data may not be properly rendered.

I am looking for a solution, and this may take some time.

thank you for the update, @longxiaofei.

1. interesting, why is this now a case - it was working back in the day (is this a new databricks limit?)
2. is there a workaround by running a f.ex. streamlit server in a databricks notebook and render it within the streamlit server? I have the issue that this works on localhost, but when running from within a streamlit app in databricks (started within a databricks notebook) it does not get rendered as well (at all) - is this also related to the ipywidget payload size?

[longxiaofei]

@ewagner70

1. interesting, I need to continue investigating this issue.
2. This is related to the sandbox environment of databricks. In local jupyter, network requests can be processed very simply, but in a third-party online notebook, it is very troublesome to be compatible with this kind of network forwarding.

[ewagner70]

@ewagner70

1. interesting, I need to continue investigating this issue.
2. This is related to the sandbox environment of databricks. In local jupyter, network requests can be processed very simply, but in a third-party online notebook, it is very troublesome to be compatible with this kind of network forwarding.

other widgets work (mitosheet, etc.) and don't seem to have any issue? Did you do something special with pygwalker? I can see that POST method is "not allowed" for whatever reason (maybe switch to GET to avoid that?)

[longxiaofei]

Hi @ewagner70, pygwalker==0.4.9.9 compresses the component code, Currently pygwalker can be used normally in databricks, you can try it.

[ewagner70]

Hi @ewagner70, pygwalker==0.4.9.9 compresses the component code, Currently pygwalker can be used normally in databricks, you can try it.

Thx @longxiaofei,

1. in classic databricks notebooks they show up now.
2. however, when running within streamlit on databricks notebook. pygwalker still doesn't show up (nothing actually is shown/started, not even an error message).