Display image in the Chain of Thought Process rather than at the end.

[tamil-acog]

Hi,

I am running an MRKL agent on my pandas dataframe. I would like to display the image generated by the agent, if any, in the Chain of Thought Process along with the observation, rather than at the end of Chain, when Final Answer is displayed.

Thanks in advance.

[tamil-acog]

Hi team, Any help here would be really appreciated. Thanks

[willydouhard]

This could be an example

[jenghub]

I reviewed the image-gen cookbook example but wasn't sure how to apply it inside the chain of thought/ @step function.

I have something roughly like this:

@cl.step(type="llm")
async def llm_generates_image():
    image_in_bytes= image_from_chain()
    im_to_send = cl.Image(content=_in_bytes, display="inline")
    await cl.Message(content="Image inside chainlit step", elements=[im_to_send]).send()
    
@cl.on_message
async def on_message():
    result = await llm_generates_image()

The image ends up displaying in the main sequence rather than in the expandable steps portion of the UI. Any suggestions? Thank you!