dosubot[bot] comments

Results 4246 comments of


                                            dosubot[bot]

"proto not found" error occured when using kubernetes proto files

Hi, @27149chen! I'm Dosu, and I'm here to help the kratos team manage their backlog. I wanted to let you know that we are marking this issue as stale. From...

[Question] How to write gorm transaction in biz directory

Hi, @iiaiaia! I'm Dosu, and I'm helping the kratos team manage their backlog. I wanted to let you know that we are marking this issue as stale. From what I...

The depth of the log package helper caller needs to be controlled separately, which is too cumbersome

Hi, @tianping526! I'm Dosu, and I'm here to help the kratos team manage their backlog. I wanted to let you know that we are marking this issue as stale. From...

Hope to add proxy options for most LLMs vendors

Based on your request, it seems you would like to extend the 'dify' software to support proxy options for other Language Learning Models (LLMs) vendors in addition to OpenAI. This...

Hope to add proxy options for most LLMs vendors

Hi, @mwnu, I'm helping the Dify team manage their backlog and am marking this issue as stale. From what I understand, the issue you raised requested the addition of proxy...

vllm模式启动，配置使用2个gpu，为什么只占用一块gpu，以下是server_config.py详细配置信息

嘿，@Andy1018！很高兴再次见到你。遇到什么新挑战了吗？根据您提供的`server_config.py`配置信息，我注意到`num_gpus`参数在`default`模型工作器的`FSCHAT_MODEL_WORKERS`配置中被注释掉了，其值被设置为`1`。这个参数直接控制要使用的GPU数量。如果要使用两个GPU，您需要取消注释这一行，并将其值设置为`2`。 ```python # "num_gpus": 1, # 使用GPU的数量 ``` 修改为： ```python "num_gpus": 2, # 使用GPU的数量 ``` 确保在您的`server_config.py`文件中，这一行是活动的（没有被注释），以启用两个GPU的使用。另外，我在Langchain-Chatchat的问题跟踪中找到了一些类似的问题，可能对您有所帮助。例如，[设置了torch.cuda.set_device(1)，不起作用](https://github.com/chatchat-space/Langchain-Chatchat/issues/681) 和 [请问如何设置用哪一张GPU？](https://github.com/chatchat-space/Langchain-Chatchat/issues/693)。这些问题中提到了一些可能的解决方案，例如直接设置设备为类似"cuda:1"的字符串，而不是调用`torch.cuda.set_device`。希望这些信息对您有所帮助。如果您有任何其他问题或需要进一步的澄清，请随时告诉我。 ## Sources - [日志显示llm device和embedding device为cuda，但实际跑的时候显卡没吃到，全由cpu在运行。](https://github.com/chatchat-space/Langchain-Chatchat/discussions/1022) - [设置了torch.cuda.set_device(1)，不起作用](https://github.com/chatchat-space/Langchain-Chatchat/issues/681) -...

vllm模式启动，配置使用2个gpu，为什么只占用一块gpu，以下是server_config.py详细配置信息

看起来您已经成功地启动了Langchain-Chatchat服务，并且使用了两块GPU进行加速。根据启动日志，服务已经在端口20000上运行，并且使用了Qwen-7B-Chat模型和bge-large-zh-v1.5的嵌入模型。如果您有任何关于Langchain-Chatchat的问题，或者需要进一步的帮助，请随时告诉我。 #### About Dosu This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point...