sglang icon indicating copy to clipboard operation
sglang copied to clipboard

Outlines v0.0.36 breaks compatibility

Open qeternity opened this issue 11 months ago • 1 comments

I don't have time to fix at the moment. We have pinned to 0.0.34 in our fork for the time being.

router init state: Traceback (most recent call last):
  File "/sglang/python/sglang/srt/managers/router/manager.py", line 68, in start_router_process
    model_client = ModelRpcClient(server_args, port_args)
  File "/sglang/python/sglang/srt/managers/router/model_rpc.py", line 619, in __init__
    self.model_server.exposed_init_model(0, server_args, port_args)
  File "/sglang/python/sglang/srt/managers/router/model_rpc.py", line 137, in exposed_init_model
    self.regex_fsm_cache = FSMCache(
  File "/sglang/python/sglang/srt/constrained/fsm_cache.py", line 8, in __init__
    self.outlines_tokenizer = TransformerTokenizer(
  File "/usr/local/lib/python3.10/dist-packages/outlines/models/transformers.py", line 63, in __init__
    self.eos_token_id = self.tokenizer.eos_token_id
AttributeError: 'str' object has no attribute 'eos_token_id'

detoken init state: init ok```

qeternity avatar Mar 19 '24 00:03 qeternity