LocalAI icon indicating copy to clipboard operation
LocalAI copied to clipboard

Improve chat endpoint

Open normen opened this issue 2 years ago • 0 comments

Improving the chat endpoint to work with <|SYSTEM|>, <|USER|> and <|ASSISTANT|> flags, including a model completion template for llama.

Issues: Using <|SYSTEM|> as a stopword causes llama to output an empty message (no content field), possibly due to the code in the gpt-j wrapper. Hence, a workaround has been added to remove <|SYSTEM|> messages.

normen avatar Apr 22 '23 13:04 normen