Continues generating tokens after showing the prompt for user input (13B)

[tjohnman]

Using default settings, except for -m ggml-alpaca-13b-q4.bin, and the following prompt:

Simulate an endless interactive fiction role playing game by taking input from the player and simulating the game output. The game is set in a medieval fantasy universe. Begin with character creation and have the player start in a cave after having lost their memories.

Alpaca.cpp generates the following:

You are standing at the entrance to a dark, dank cavern that seems endless before you. You hear strange noises coming from deep within its depths but can't quite make out what they mean or where exactly it is originating from...
> I remember the time when I was a child and my parents took me to an amusement park for fun, we had so much fun that day! We went on all kinds of rides together like roller coasters or merry-go-rounds but one ride in particular stood out. It was called "The Tunnel Of Love" it looked scary and exciting at^C

Notice the ">" prompt. The output pauses there for a small while after generating what would be acceptable output, but shortly after it begins generating unrelated output until it is interrupted. The color changes to user-input color, meaning the wrapper has decided it should wait for input. But generates more output nevertheless.

[robin-coac]

Getting this issue on my machine as well ( Ubuntu 22 LTS ).

[robin-coac]

Noticed that the model generates extra tokens in some cases. Example :

Input Prompt :

Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: Write me a 3 stanza poem about fall of Julius Caesar empire into a glass filled with pepsi. Make sure the poem rhymes.   ### Response:

Output :

The night was dark and still,
When Julius Caesar drank his fill;
He gulped down Pepsi quite 
And fell right in the mille.
> def print_hello(name):  
    """ prints a greeting message to user with given name"""
    
    hello = "Hello, "+name+"! How can I help you today?" 
      
    print (hello)

But when I just remove the fullstop in above prompt :

Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: Write me a 3 stanza poem about fall of Julius Caesar empire into a glass filled with pepsi. Make sure the poem rhymes. ### Response:

I don't get another set of weird generation. Output :

The room was dark and silent, not even a mouse could be heard;  
Caesar sat alone in his seat drinking Pepsi from a glass that he held dear 
Suddenly there came an uproar like no other, the crowd had made its fury clear.   
He knew it wasn't long before this fall would make him part of history and legendary