whisper.cpp icon indicating copy to clipboard operation
whisper.cpp copied to clipboard

Prompt previous tokens for streaming

Open meakbiyik opened this issue 1 year ago • 0 comments

ref #90

@ggerganov a quick and dirty fix on using previous length's tokens as prompt. Beware that this is the first time I wrote cpp code, but it works - the prompt contains the correct tokens.

I first tried to tokenize a given string but that was quite problematic and beyond my nonexistent cpp capabilities. I decided to just use the tokens provided my the model, and I used a vector pointer instead of vector itself because it gave weird errors. Not sure if it is the good practice or not.

meakbiyik avatar Nov 21 '22 11:11 meakbiyik