sglang icon indicating copy to clipboard operation
sglang copied to clipboard

AttributeError: 'str' object has no attribute 'eos_token_id'

Open Yanshang1991 opened this issue 11 months ago • 6 comments

INFO 03-13 07:48:22 weight_utils.py:163] Using model weights format ['*.safetensors'] Rank 0: load weight end. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Rank 0: max_total_num_token=52678, max_prefill_num_token=32768, context_len=32768, disable_radix_cache=False, enable_flashinfer=False, disable_regex_jump_forward=False, disable_disk_cache=False, attention_reduce_in_fp32=False router init state: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/router/manager.py", line 68, in start_router_process model_client = ModelRpcClient(server_args, port_args) File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/router/model_rpc.py", line 619, in init self.model_server.exposed_init_model(0, server_args, port_args) File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/router/model_rpc.py", line 137, in exposed_init_model self.regex_fsm_cache = FSMCache( File "/usr/local/lib/python3.10/dist-packages/sglang/srt/constrained/fsm_cache.py", line 8, in init self.outlines_tokenizer = TransformerTokenizer( File "/usr/local/lib/python3.10/dist-packages/outlines/models/transformers.py", line 63, in init self.eos_token_id = self.tokenizer.eos_token_id AttributeError: 'str' object has no attribute 'eos_token_id'

I encountered an error while running the Qwen model with sglang=v0.1.13 version.

Yanshang1991 avatar Mar 13 '24 07:03 Yanshang1991