Add interactive plots within gr.Chatbot

[Gabriel-Macias]

• [x] I have searched to see if a similar issue already exists.

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
I'm currently looking into building a chat application that is able to show certain plots to users based on their requests and was wondering if is possible to embed other gradio components such as gr.Plot or some way to include interactive plots made via plotly or bokeh within the chatbox as opposed to including a separate gr.Plot component to display them outside of the chat. My current solution only considers inserting static images in the chatbot via hyperlinks.

[abidlabs]

Hi @Gabriel-Macias this is an interesting idea, but probably not one we'll support in the core library soon as it would require a significant refactoring of gr.Chatbot. I do see the value of it down the line, so let's keep this issue open

[trivikramak]

Hi @abidlabs, Is there any update on the timeline or priority of this feature/enhancement?

[abidlabs]

Not yet -- I think this will require quite a bit of work unfortunately. cc @dawoodkhan82

[codenprogressive]

Hi @abidlabs any chance this feature could be implemented? I see a lot of value having it part of Chatbot capabilities!

[freddyaboulton]

I think this could be an interesting custom component!

[andreped]

This would be of interest to me as well. We are able to do this in Streamlit without any additional libraries.

Does anyone know of a custom gr.Chatbot widget, which offers support for adding "any" gradio component such as gr.Dataframe or gr.Code inside a chat bubble?

[codenprogressive]

@freddyaboulton any snippet code you can share on how we can integrate plotly graphs in gr.Chatbot?

[freddyaboulton]

Hi @liwalayouni ! I'm not sure what the general solution looks like because there is no standard format for an LLM to specify a plot. For the plot case, I think you can handle this in the prediction function.

Parse the LLM output for a plot, save it to an image and then embed the image using html. Something like this <img src='/file={image}' /> where image is the image path.

[codenprogressive]

Hey @freddyaboulton assuming the LLM generated the chart and we have a Ploly Figure Object. The ask was how to plot it as interactive in the gr.chatbot. i.e can gr.Chatbot integrates a subcomponent like gr.Plot?

[freddyaboulton]

No built-in support. For now, I think you need to embed as an image or if you want interactivity, saving to html and embedding as an iframe in the response may work (have not tried)

https://plotly.com/python/interactive-html-export/