sglang
sglang copied to clipboard
Outlines v0.0.36 breaks compatibility
I don't have time to fix at the moment. We have pinned to 0.0.34 in our fork for the time being.
router init state: Traceback (most recent call last):
File "/sglang/python/sglang/srt/managers/router/manager.py", line 68, in start_router_process
model_client = ModelRpcClient(server_args, port_args)
File "/sglang/python/sglang/srt/managers/router/model_rpc.py", line 619, in __init__
self.model_server.exposed_init_model(0, server_args, port_args)
File "/sglang/python/sglang/srt/managers/router/model_rpc.py", line 137, in exposed_init_model
self.regex_fsm_cache = FSMCache(
File "/sglang/python/sglang/srt/constrained/fsm_cache.py", line 8, in __init__
self.outlines_tokenizer = TransformerTokenizer(
File "/usr/local/lib/python3.10/dist-packages/outlines/models/transformers.py", line 63, in __init__
self.eos_token_id = self.tokenizer.eos_token_id
AttributeError: 'str' object has no attribute 'eos_token_id'
detoken init state: init ok```