Jeffser
Markdown rendering before response generation stopped
Is your feature request related to a problem? Please describe. Often responses contain copyable blocks of information, while the response generation is on going the user can not easily copy that information.
Describe the solution you'd like Detect closure of markdown parts and render them while the generation is on going. I personally would prefer to only get blocks of markdown served instead of an on going text generation with a final markdown rendering.
I'm not sure about the implications regarding the UI if it will look suitable.
I would like to hear the opinions of other users. Maybe it's just me.
I believe the implementation of this would actually be possible fairly well, however it could lead to lag especially on older machines because Adwaita has to draw the full UI changes every (other) time some tokens arrive. Some setting will need to be provided in order to make that optional.
Hi, I thought I responded to this but apparently I didn't.
Gtk4 is kinda weird in regards to markdown rendering, for starters, it doesn't actually support markdown at all, it uses something called Pango Markup.
The way Alpaca renders it's "markdown" is by converting it once the message finished generating by replacing tags like this:
# title -> <span size="xx-large">title</span>
**bold** -> <b>bold</b>
`mono` -> <tt>mono</tt>
etc etc
Now let's imagine how this would work when a model is generating text
# ti -> <span size="xx-large">ti</span>
tle -> <span size="xx-large">title</span>
**t -> <b>t</b>
ext is -> <b>text is</b>
bold** -> <b>text is bold</b>
As you can see it's a lot of going back forward guessing what the model wants, some models might even output whole paragraphs with multiple markdown stuff to manage.
And then Alpaca would have to also manage the rendering of live tables, code blocks, latex, thought blocks, etc
Now another different problem is the lag, when generating text Alpaca uses a custom widget based on GTKTextView which is made to handle constant updates but manages markup completely differently.
Then when the magic happens and everything gets rendered properly it uses another custom widget based on GTKLabel which works perfectly with Pango Markup but lags the app completely if it's updated constantly.
TLDR: Harder than it looks
