bug: broken image & audio in reply from model

[hwc0x01]

Describe the bug when asking a model to generate an image/audio the reply comes back with a broken/non-displaying image or broken audio player

Steps to reproduce Steps to reproduce the behavior: see screenshots

Expected behavior expect either a displayed image/audio file or a link to an image/audio saved locally -- both should have options to save as

Screenshots

Environment details

Operating System: Ubuntu 22.04.01
Jan Version: 0.4.3-118
Processor: Intel® Core™ i5-2520M CPU @ 2.50GHz × 4
RAM: 16GB

Additional context Add any other context or information that could be helpful in diagnosing the problem.

[hwc0x01]

also causes ui display issue (see gif) for following prompt

[hwc0x01]

audio generation has a dead player (probably because file doesn't exist) but it's a broken output behavior

[namchuai]

@hwc0x01 , could you please send me the zip of those threads?

It can be located under ~/jan/threads

[hwc0x01]

thread.json problemthread.txt

[namchuai]

The image url for the message is https://www.pexels.com/photo/mountains-landscape-4167306/pexels-photo-4167306.jpeg, which I can't open as well.

I tried to replace the URL with lorem picsum's link and Jan can display normally.

However, I think we need to have better UI for URL we failed to fetch.

[hiro-v]

Hi there, I believe at this stage (0.4.3 and 0.4.4) Jan app has not officially supported image/ audio generation. Given the 2 above comments, I think it's caused by the LLM output which can be malformed/ outdated. I think this is the expected behavior at the moment, not bug.

Thank you.