Alexandre Strube

> The repo is currently private since we're not sure where our custom kernels should live yet. What environment are you running in (OS, presence of CUDA gpus, python version,...

I get this with the latest sglang: ``` object address : 0x150df025af80 object refcount : 4 object type : 0x151167c6a320 object type name: RecursionError object repr : RecursionError('maximum recursion depth...

> ```python > if __name__ == "__main__": > sys.setrecursionlimit(8000) > parser = argparse.ArgumentParser() > ServerArgs.add_cli_args(parser) > args = parser.parse_args() > server_args = ServerArgs.from_cli_args(args) > launch_server(server_args, None) > ``` Tried, I...

With regard to @mtekman 's comment, many other providers have a common provider called OpenAI API or equivalent. It uses the same "openai" python package, with the difference that it's...

> > ```python > > os.environ['OPENAI_API_BASE']="http://0.0.0.0:5001/v1" > > ``` > > the default base url is http://0.0.0.0:5000 , how can i change it ? That seems to be the issue...

There is one merge request going around which provides support for multi modal models. Are you sing the version from pip or a git clone of the ain repo? I...

I would recommend you to remove the one installed with pip and use the one directly cloned from the repo, and keep it up to date with git pull. Things...

I have used dozens of models from huggingface. Those are the pip requirements which I install in the venv: ``` pip torch sglang[srt] accelerate flash-attn transformers @ git+https://github.com/huggingface/transformers.git mamba-ssm causal-conv1d>=1.2.0...

``` Incoming data... searched domain:captive.apple.com. Response captive.apple.com. == 10.0.0.1 b'\x96\x02\x81\x80\x00\x01\x00\x01\x00\x00\x00\x00\x07captive\x05apple\x03com\x00\x00A\x00\x01\xc0\x0c\x00\x01\x00\x01\x00\x00\x00 10.0.0.1 Incoming data... searched domain:captive.apple.com. Response captive.apple.com. == 10.0.0.1 b'\x96\x02\x81\x80\x00\x01\x00\x01\x00\x00\x00\x00\x07captive\x05apple\x03com\x00\x00A\x00\x01\xc0\x0c\x00\x01\x00\x01\x00\x00\x00

Can it be related to the two errors shown up there? The one about not launching the async task and the NameError exception?