feat: notebook mode (text completion)

[robertvazan]

Is your feature request related to a problem? Please describe. Ollama and cloud APIs support simple text generation (continuations) in addition to chat, but this is not exposed in WebUI.

Describe the solution you'd like Maybe start with something like this: https://github.com/oobabooga/screenshots/raw/main/print_default.png

With some shortcuts (tab to accept next word from continuation) and automatic regeneration when the prompt changes, this would allow for fairly convenient assisted writing. KV cache will keep it performant unless you go back and edit something at the beginning of the prompt.

Another option is to offer several short continuations.

Describe alternatives you've considered Instructing the model to continue the story is unwieldy, requires instruction-tuned model, and works poorly with smaller models. It's possible to use a specialized tool, but I already have Ollama & WebUI set up (I have not researched the market comprehensively yet, so not sure what's available).

[jannikstdl]

Hi, have you tried this?

[robertvazan]

@jannikstdl That's not what I have in mind. That button just encourages the LLM to write a longer response. I need the LLM to suggest the next few words for what I am writing. Base models are intended to be used like that. I could edit the response to contain beginning of what I am writing and then request continuation, but that's impractical unless the LLM is able to predict long stretches of text reliably, which the small ones cannot do. I need something more streamlined. The other issue is that this uses instruction-tuned model, which will insist on being an assistant, whereas (I hope that) the base models available in Ollama are more flexible.

[robertvazan]

I am thinking of something akin to NovelAI.

[jannikstdl]

I think this is something like the mention feature. But as far as i know its WIP.

[robertvazan]

@jannikstdl Still not it. I don't want to produce a conversation between two AIs/characters. I want to write arbitrary text and have the AI suggest next few words / list items / even whole paragraphs. Try the free NovelAI (no registration) to see what I mean.