Robin Chen

@649453932 这是我的requirements.txt，可供参考 ``` boto3==1.20.54 botocore==1.23.54 certifi==2021.10.8 charset-normalizer==2.0.12 idna==3.3 jmespath==0.10.0 joblib==1.1.0 numpy==1.22.2 python-dateutil==2.8.2 regex==2022.1.18 requests==2.27.1 s3transfer==0.5.1 scikit-learn==1.0.2 scipy==1.8.0 six==1.16.0 sklearn==0.0 threadpoolctl==3.1.0 torch==1.10.2 tqdm==4.62.3 typing-extensions==4.1.0 urllib3==1.26.8 ```

> I am trying to use the dspy.HFModel object to load models locally. I keep getting > > ``` > ValueError: temperature has to be a strictly positive float, but...

> @IcyFeather233 It's fixed in the DSPy version that's in main, but there hasn't been a release yet. So you could clone and install from the repo. Maybe it's enough...

按照这种方式修改，翻译arxiv论文的插件似乎不能多线程翻译，只能一条一条翻译，需要再加上这个配置：https://github.com/binary-husky/gpt_academic/issues/1762

To complete this issue, does it mean that I need to have the corresponding GPU resources to run large models for project debugging? Additionally, I am aware of an outstanding...

Hi! To complete this issue, does it mean that I need to have the corresponding GPU resources to run large models for project debugging?

I have a question about the Benchmark Dataset Map: which domains should this dataset cover? Is it for all domains, or just industrial and government sectors? Also, if I need...

/assign

我感觉这个应该是要看VLLM的文档，：https://docs.vllm.ai/en/latest/serving/distributed_serving.html，我tensor_parallel_size设置的和GPU数量一样是可以的。

我也是有同样的问题，有的模型可以把显存吃满，比如我用Qwen14b-chat就可以把8*A800差不多吃满，但是有的模型，比如autoj-bilingual-6b，我的八卡每张卡的占用率还不到10%，我怀疑这个是跟模型有关 另外，感觉开启VLLM会让占用率变高推理速度变快，如果是使用HuggingFaceCausalLM占用率和速度都会低一些