zouchl

用gunicorn 部署embedding模型服务，采用workers > 1时，进程会死掉重启。

1

3090 显卡上。 使用workers = 1，可正常运行

直接部署embedding，在调用API时，如果请求数稍微大一点，model.encode(query_list)报NCCL Error 3错误

1

cuda11.8 12.2都存在这个问题， embeddings = model.encode(queries) File "/usr/local/python3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/usr/local/python3/lib/python3.10/site-packages/FlagEmbedding/flag_models.py", line 90, in encode last_hidden_state = self.model(**inputs, return_dict=True).last_hidden_state File "/usr/local/python3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl...