jefffortune

@haliliceylan if I use Mistral Instruct 7B Q4 from Jan.ai model hub it works as expected. If I provide the OpenAI settings with my URL the hosted env in AWS...

@Van-QA I downloaded the nightly fix and try it out. My steps: - Jan nightly v0.4.11-392 - I am using VLLM as the openai compatible server - I used a...

when using ```python streamer = TextIteratorStreamer( tokenizer, skip_prompt=True, skip_special_tokens=True ) ``` I don't see any streaming at all, no matter the code updates that follow. When I switched to following...

@eyurtsev, thanks for the advice on code cleanup. This simplification does not return dict anymore but the generated text on the client.py. I will provide the updated code to show...