Disable streaming i.e. in replace mode to allow Ctrl+Z

[DVLP]

Currently once the answer comes back it takes a lot of undo operations to just to revert. There should be an option to disable streaming globally or for certain modes.

[yaroslavyaroslav]

Fair point, have suffered from this myself, so it will be implemented in some of nearest releases.

Note for future me: the easy part is to add stream property to AssistantSettings struct, the hard part is to add non streaming response handling, which is missed in current plugin implementation.

[yaroslavyaroslav]

A thoughts in update to this issue. I'm considering a non streaming solution for such as a bad one: if there's a network lag occurring (there's a very few of them for the gpt-4o at the time being, but this isn't the fact for most of third party services, where both model prompting pace and network delay can be significant) it leaving a user within a quite uncomfortable state, when he/she is unable to determine whether it's safe to use the view which he/she focused on now or not. Meaning it's completely opaque does the request already failed completely or is it pending, and changing view state would break things.

This thing could be acceptable if it was a read only view (output view, separate read-only chat tab), but it's a far from being convenient with the ordinary view where he/she content presented.

The compromise solution for that issue, more like work around tbh, is to set a view checkpoint before streaming execution, and to provide a separate contextual binding for undo them all at once for the cmd/ctrl+z (contextual means, that this command wouldn't be performed out of this very strict conditions), but this is a relatively effort extensive solution to implement, so it's unlikely that I would take it into work soonish.

[IPWright83]

I think some sort of Sublime Text loading indicator along the bottom bar would be sufficient to tell you "I'm working on it"?

[fruffy]

I think some sort of Sublime Text loading indicator along the bottom bar would be sufficient to tell you "I'm working on it"?

Agree, I have been running to the Ctrl+Z issue a lot, particularly when prompts are not working out as I'd like to. I think even just as an option that one can toggle (and the option has a warning about problematic behavior).

[yaroslavyaroslav]

Let me be clear here: the problem is that in the current state of the code there's only streaming response handler implemented. So it's not just about the toggle, it's about updating network layer and the related presentation logic, which is out of my plans for the time being.

The good news here, that there's the solution for that under the development: phantom (like lsp popup, but inlined into view) streaming presentation with actions to copy, append and replace selected buffer content (latter two actions to be implemented).

The only caveat that I'm facing while developing (and using) it is that phantom itself has no line wrapping isolated option, so you have to toggle word wrapping for the whole view to make it convenient to work with say markdown output.

[DVLP]

and now donation begging added to every response lol

Here's a working POC how to disable streaming. It means it may take a while to see the response if it's long https://github.com/yaroslavyaroslav/OpenAI-sublime-text/pull/60/commits/c8ca02c49665da6976969a49b056784d0e85cb8f also needs setting stream = False in the other file

[yaroslavyaroslav]

Thank you for your contribution @DVLP

I've extended the POC to its full capability, so in the next release there will be streaming toggle. Can't promise that I'll release it right away, but it'll be there within the month at its worst, that's for sure.

[adamserafini]

Can I suggest an alternative to Ctrl-Z that solves this issue IMO? Use the inbuilt "revert_hunk" command in Sublime, which by default is bound to ["super+k", "super+z"], at least on my Mac install.

So, instead of ⌘-Z, do this: ⌘-K, ⌘-Z (Command K, and then Command Z after), while the cursor is in the new hunk that was just written by the LLM. This also has the benefit that reverting the diff is not "all or nothing", if there are multiple hunks they can be accepted or rejected with this method.

It's more of a side point to this issue, but the far greater usability issue to me is that in the "replace" case, the written code is quoted and explained so isn't immediately usable without further editing, ie. instead of getting this in the sheet:

<!DOCTYPE html>

I get this

```html
<!DOCTYPE html>

And some verbose blah blah from the LLM here.

Anyone know a workaround for that problem? (I am perhaps ambituously trying to emulate a "cursor-like" UX in Sublime).

[yaroslavyaroslav]

@adamserafini this is major llm flaw actually, I'm facing exact this even in zed inline assisting mode, despite they're wrapping each of such request with additional prompting to force model response within special token.

The general issue here and in zed that in sake of providing to llm additional context all selected text is automatically wrapped in '''language_name \n\n\n''' tags in request, which forces the already tempted to answer in markdown model to do that even more.

Actually I think this is where prompt engineering comes out, you have to order the model to not wrap its answers with anything. But again even zed fails with this sometimes. You can literally ask llm to provide you the appropriate prompt for that though.