Output with --verbose prints previous response only after next prompt entered

[dblacknc]

Describe the bug

I expect when adding --verbose to the server.py args, prompts and responses would be printed after being issued, and preferably not duplicated but I get the set of them is the conversation context. I'm thinking it's not working as intended. If what it's doing is intentional, I guess this bug can be closed as a "works as designed."

Instead what I see on the first iteration is Just my prompt, then the "Output generated..." info. On the next iteration the first response is finally seen, along with the next prompt. That is, the previous response is only logged with the next prompt.

Also, at each prompt the entire conversation to that point (the prompt and before it), is printed. Maybe this is intended, for context. Wouldn't it make more sense to just print the previously entered prompt or displayed response?

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Add --verbose and start a chat. Enter a prompt and wait for the response. The prompt is immediately printed, and at the end of the response (seen in the UI) the "Output generated..." info is seen, but not the response text.

Enter another prompt. Then the previous response shows, along with all prompts and responses previously entered (duplicated).

Screenshot

No response

Logs

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
What's the terminal airspeed velocity of an unladen swallow, when in a dive?

### Response:

--------------------

Output generated in 50.45 seconds (0.69 tokens/s, 35 tokens, context 58, seed 2026866583)


Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
What's the terminal airspeed velocity of an unladen swallow, when in a dive?

### Response:
The terminal airspeed velocity of an unladen swallow, when in a dive, is approximately 24 miles per hour (38 kilometers per hour).

### Instruction:
What are the factors that determine this dive velocity?

### Response:

--------------------

Output generated in 65.64 seconds (1.10 tokens/s, 72 tokens, context 119, seed 956254431)


Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
What's the terminal airspeed velocity of an unladen swallow, when in a dive?

### Response:
The terminal airspeed velocity of an unladen swallow, when in a dive, is approximately 24 miles per hour (38 kilometers per hour).

### Instruction:
What are the factors that determine this dive velocity?

### Response:
There are several factors that can affect the terminal airspeed velocity of an unladen swallow during a dive, including wind speed and direction, altitude, temperature, humidity, and the bird's mass and wing shape. Additionally, the angle at which the bird enters the dive also plays a role in determining its final velocity.

### Instruction:
What is the optimal angle for the bird to enter the dive, to achieve maximum velocity?

### Response:

--------------------

Output generated in 51.01 seconds (0.90 tokens/s, 46 tokens, context 225, seed 707114328)

(The last response is displayed in the UI but not logged here - yet.)
FYI it was this:

"For maximum velocity, the bird should enter the dive at an angle between 15-20 degrees from horizontal. This allows it to take advantage of the aerodynamic forces generated by its wings as it descends."

System Info

Ubuntu 22.04
RTX 3060
Model loaded: gozfarb/llama-13b-pretrained-sft-do2-4bit-128g
Instruct mode, either Alpaca or None template behaves same

[kuso-ge]

It is intended. The reason your conversation history is printed every generation is because they are used as context. When you reach max context, you'll notice that the oldest prompt/responses are removed from the context history. These way, you'll know which lines of text are removed and which are retained as context.

[dblacknc]

The bug report is response N is printed in the verbose log only after prompt N+1 is entered.

Sorry for any confusion from my also mentioning the entire history is printed each time. I understand why that's needed and intended.

Obviously at some point I'm going to stop the conversation either by abandoning the chat, clearing the history, or clicking Stop... all these likely after a response. The current logging behavior is that last response N is never printed. Or put another way, in the log at any one time, only response N-1 is ever visible.

[github-actions[bot]]

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.