Alexandre Strube
Alexandre Strube
```python import os import sys from dask_mpi import initialize from dask.distributed import Client import joblib if len(sys.argv) != 2: print("Usage ", sys.argv[0]," ") sys.exit() else: N = int(sys.argv[1]) def f(x):...
This also does not work: https://support.nesi.org.nz/hc/en-gb/articles/360001392636-Configuring-Dask-MPI-jobs
I'm having difficulty having a working example of dask.
Adding the model with the register_worker.py will fail with a 404. For example, an ollama server is running at its default port 11434. If I add it via the json...
> Have you found the solution for this issue? I'm having it too Either fastchat fixes the vllm_worker or someone sends a fix, or you deploy a different system, like...
@vikrantrathore @aliasaria @Michaelvll @zeyugao @yaofeng @rudeigerc @pandada8 @infwinston @congchan @merrymercy This is an important one to keep FastChat as an useful inference platform
Fix is provided on the pull request. @LizLizLi you can simply set `VLLM_USE_V1` prior to running fastChat to 0
The error `2025-04-08 17:50:17 | ERROR | stderr | AttributeError: 'AsyncLLM' object has no attribute 'engine'` happens on vllm V1 when using Numpy 1.x which some dependency brings. When changing...
@zeyugao this is the approach we should be using here all along, to be honest. I will check your repo
I'm getting this with model_worker.py . sglang_worker doesn't even load it. ``` 2024-07-11 17:52:19 | INFO | stdout | INFO: 134.94.0.1:33456 - "POST /worker_generate_stream HTTP/1.1" 200 OK 2024-07-11 17:52:21 |...