Running PyGWalker in a Hugging Face space

[severo]

It would be amazing to test PyGWalker in a Hugging Face space, particularly on datasets hosted on the Hub.

[longxiaofei]

Yes! I create new space base streamlit, quickly paste pygwalker demo to this space. Then, I released a higher performance streamlit server, I haven't tried import dataset on hub yet.

https://huggingface.co/spaces/Libra-1995/test-pyg

[ObservedObserver]

It would be amazing to test PyGWalker in a Hugging Face space, particularly on datasets hosted on the Hub.

That's a great idea. Besides running a pygwalker app for a fixed dataset.

I am curious about is there a more native or integrated way to embed a pygwalker app for exploring any datasets in hugging face.

My ideal ways of using pygwalker with huggingface are:

1. You can open it directly when viewing the dataset, like a place in the dataset viewer or a tab.

2. Using hugging face API to allow users to start a pygwalker workspace with any dataset in hugging face directly. The workspace can be native embedded in hugging face, a online co-lab, steamlit app, or kanaries hosting app.

[severo]

nice @longxiaofei! Possibly we could select the dataset (plus config and split) we want to visualize, using controls. As proposed by @ObservedObserver (option 2), you can get the list of datasets from the HF API, and if you want to dig into more details (configs, splits and more), you have the Datasets Server API.

I don't know Streamlit but maybe it's easy to do with Gradio. @freddyaboulton @abidlabs do you have examples of such spaces?

---

@ObservedObserver on your point 1: it's a very good idea. It requires integration with the Hub, so it's more long-term, but we will think about it. It's something we have been asked for already.

[julien-c]

What about adding a badge like https://shields.io/ to dataset cards that opens a Space with the corresponding dataset_name as a query parameter? Would that work?

[longxiaofei]

good ideas, I'll explore Hugging Face space and Datasets Server API in the next two days.

[abidlabs]

Nice! In terms of using Gradio with Pygwalker, we can embed any sort of HTML using the HTML component. So you could do for example:

with gr.Blocks() as demo:
   gr.HTML("<iframe src=...></iframe>")
demo.launch()

We're also about to release a way to create custom Gradio components, which would be interesting to explore with Pygwalker. Let me know if there's interest in exploring more.

[ObservedObserver]

Nice! In terms of using Gradio with Pygwalker, we can embed any sort of HTML using the HTML component. So you could do for example:

with gr.Blocks() as demo:
   gr.HTML("<iframe src=...></iframe>")
demo.launch()

We're also about to release a way to create custom Gradio components, which would be interesting to explore with Pygwalker. Let me know if there's interest in exploring more.

Yeah, we are pretty interested in exploring it. For the code above, it looks similar to Streamlit, I suppose it will work fine since pygwalker works there. One problem that might be worried about is that pygwalker is not just a piece of HTML, it contains computation services in python kernel, which means it requires the framework to provide a communication system between client and backend.

[longxiaofei]

What about adding a badge like https://shields.io/ to dataset cards that opens a Space with the corresponding dataset_name as a query parameter? Would that work?

I will quickly develop a POC.

[longxiaofei]

@julien-c, kanaries can provide server support.

like this? a simple demo:

[ObservedObserver]

Nice! In terms of using Gradio with Pygwalker, we can embed any sort of HTML using the HTML component. So you could do for example:

with gr.Blocks() as demo:
   gr.HTML("<iframe src=...></iframe>")
demo.launch()

We're also about to release a way to create custom Gradio components, which would be interesting to explore with Pygwalker. Let me know if there's interest in exploring more.

I just wrote a tutorial of how to use pygwalker in gradio, it's pretty simple for the basic usage. https://docs.kanaries.net/pygwalker/tutorials/use-pygwalker-in-gradio

But pygwalker has a advance feature that allows users to enable a duckdb engine in python kernel(which can be much faster for computations). I am wondering if it is possible gradio can support the communication between the front-end and back-end to enable this feature in pygwalker.

[abidlabs]

Awesome! As I mentioned beofre, we're working on making it possible for Gradio users to create their own custom components -- meaning that you'll be able to take an existing Gradio component and clone it, modify the backend or the frontend, and use it in your Gradio apps. If you're interested, I can share with you the current instructions for making a custom component and you could give it a shot. What do you think?

[ObservedObserver]

Awesome! As I mentioned beofre, we're working on making it possible for Gradio users to create their own custom components -- meaning that you'll be able to take an existing Gradio component and clone it, modify the backend or the frontend, and use it in your Gradio apps. If you're interested, I can share with you the current instructions for making a custom component and you could give it a shot. What do you think?

That's great, would love to have a try!

[abidlabs]

Ok awesome! Here's instructions that we put together: https://github.com/gradio-app/gradio/wiki/%F0%9F%8E%A8-How-to-Make-a-Gradio-Custom-Component

Please let me know if you have any questions, happy to help!

[longxiaofei]

@abidlabs Hi, Thank you very much for your support.

I have an idea to quickly connect Pygwalker to Gradio.

Most of pygwalker's communication can be done through simple http(request-response).

Is there any way for me to add a custom route to Fastapi of Gradio?

I try to hack it.(BTW, Why is the main thread blocked when prevent_thread_lock=False?).

import gradio as gr

def custom_router():
    return {"message": "hello world"}

with gr.Blocks() as demo:
    gr.HTML("hello world")

result = demo.launch(prevent_thread_lock=True)
result[0].add_api_route("/pyg_custom_route", custom_router, methods=["GET"])
demo.block_thread()

Or, use app_kwargs? example:

import gradio as gr

from starlette.routing import Route
from starlette.responses import JSONResponse
from starlette.requests import Request


def custom_router(_: Request):
    return JSONResponse({"message": "hello world"})


with gr.Blocks() as demo:
    gr.HTML("hello world")


result = demo.launch(app_kwargs={
    "routes": [Route("/custom_route", custom_router, methods=["GET"])]
})

Or, maybe add a feature: mount_fastapi_app, similar to mount_gradio_app, example:

...
mounted_app = Fastapi(...)

demo.launch(
    mounted_fastapi_app=mounted_app
)
...

def launch(...):
    ...
    app = App.create_app(blocks, app_kwargs=app_kwargs)
    if mounted_fastapi_app:
        app.mount("xxx", mounted_fastapi_app)

Maybe there are other ways? Which method do you recommend?

Looking forward to your reply.

[freddyaboulton]

Hi @longxiaofei ! Are you trying to create a custom component for PyGWalker?

If you want to send http requests from your front-end directly to your component backend, you can use the @server decorator to expose a method of your component class as an http route that your front-end code can call.

We have this working in the FileExplorer component

• @server decorator: https://github.com/gradio-app/gradio/blob/66549d8d256b1845c8c5efa0384695b36cb46eab/gradio/components/file_explorer.py#L150
• Using it in the front-end: https://github.com/gradio-app/gradio/blob/66549d8d256b1845c8c5efa0384695b36cb46eab/js/fileexplorer/Index.svelte#L31