wuriyanto

we face the same issue, is there a solution for this problem? thank you

we face the same issue, is there a solution for this problem? thank you

we are experiencing the same issue when the throughput is high Kafkajs: 2.2.4 ``` {"level":"ERROR","timestamp":"2023-10-19T11:59:31.955Z","logger":"kafkajs","message":"[Consumer] Crash: KafkaJSNumberOfRetriesExceeded: Request Fetch(key: 1, version: 11) timed out","groupId":"consumer-group-288578","retryCount":5,"stack":"KafkaJSNonRetriableError\n Caused by: KafkaJSRequestTimeoutError: Request Fetch(key: 1,...

i just noticed, there is no `` token in the`phi3's chat_template` config. https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/blob/main/tokenizer_config.json adding this config to the tokenizer, it works ```python tokenizer.use_default_system_prompt = True tokenizer.chat_template = "{{ bos_token }}{%...

Thank you @CaptXiong , for pointing it out

@rb81 You don't need to recompile the source code, just like this ```python model_id = "microsoft/Phi-3-mini-4k-instruct" model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype="auto", trust_remote_code=True, ) tokenizer = AutoTokenizer.from_pretrained(model_id) messages = [ {...