MagicSource

@peakji second one is, I fisrt usign model = StreamModel(model, tokeizer) and then using model.tokenizer to decode. Can u guys provide a effect print correct values without change line demo?...

@peakji thank u! I have solved the first problem. the english seems oK now. but Chinese still not OK   the Chinese characters some are ok, some still got...

Some \n which is actually needed seems trimed: 

I resolved the \n issue, but clearly the Chinese not always work:  Please take a deeper test!

语音合成的音频例子，我想听一下看看效果如何

Hello, would like address some decode on Japanese character with wrong when decode in streaming. Does anybody can help how to fix it? (this happened because of llama tokenizer vocab...

@marella Sure, this is the code: ```python llm = AutoModelForCausalLM.from_pretrained(m_f, gpu_layers=150) conv = get_default_conv_template(args.conv_template) history = [] while True: qs = input('> ') conv.append_message_single_turn(qs) prompt = conv.get_prompt() if args.debug: print(prompt)...

@matthoffner Hello, where is your code snippet using ctransfomers as openai-like server backend? Would like use it.

@matthoffner thanks so much, just found it in files tab. However, I found some weried issue. The detokenized text return to my client dropped space:  I changed nothing but...

@matthoffner Can u tell me how did u resolve the white space issue? I printted out in stream one by one, it actually didn't have white space between. Meanwhile, it...