Zhangquan Chen comments

Results 11 comments of


                                            Zhangquan Chen

[BUG] vllm推理qwen-72b-chat返回异常

遇到了同样的问题，请问解决了吗

Question for the dataset and the inference

Some question, how can I infer the process?

Qwen2-72B-Instruct-gptq-int4重复问题

> 机器A800，vLLM 0.5.0，prompt是开始，输出max tokens=2048，temperature设0.7 > > vLLM加载Qwen2-72B-Instruct-gptq-int4，使用vLLM的benchmark脚本来做并发测试，无论是1个并发限制还是10个并发限制，输出均会重复。 https://github.com/vllm-project/vllm/blob/main/benchmarks/benchmark_serving.py > > ![企业微信截图_1718760902864](https://private-user-images.githubusercontent.com/57557769/340890300-380b10a9-77bf-429c-9e04-1e528fd5e43c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA3MDM0MzksIm5iZiI6MTcyMDcwMzEzOSwicGF0aCI6Ii81NzU1Nzc2OS8zNDA4OTAzMDAtMzgwYjEwYTktNzdiZi00MjljLTllMDQtMWU1MjhmZDVlNDNjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzExVDEzMDUzOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVmODBkNGU2ZmE3NzM3ZjZiYzEwN2Q3ODdkYjUwMGI5MTIzMTlhMWE3MzkzZmY0M2NiOWE4Mjk0ZWQyNTJiOTUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.poU6biU-x0Kqj7ZoAdv-LfIV00OgA-RPX7ddLIPk7pk) > > ![企业微信截图_1718760893772](https://private-user-images.githubusercontent.com/57557769/340890330-b1b261da-464f-4d86-b5ae-08eb54ce806f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA3MDM0MzksIm5iZiI6MTcyMDcwMzEzOSwicGF0aCI6Ii81NzU1Nzc2OS8zNDA4OTAzMzAtYjFiMjYxZGEtNDY0Zi00ZDg2LWI1YWUtMDhlYjU0Y2U4MDZmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzExVDEzMDUzOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWQzOWRhNzNmMjViNDE3YmY1OGMyMzlkNTExMWFkNzQwNTgyYTYyNDI4MWVlMGYxYjI3Y2ViMzYwM2Y2NTNiNTYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.0lBtAWFVyOc9PlTVxYcc7z0mjZiP8Y3MRS8ldfNPWIk) > > 当然我也测试了无限制并发的情况下，也会生成重复 > > ![image](https://private-user-images.githubusercontent.com/57557769/340891602-5da613c0-f9e1-4b2f-bf4e-85d28c720bc8.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA3MDM0MzksIm5iZiI6MTcyMDcwMzEzOSwicGF0aCI6Ii81NzU1Nzc2OS8zNDA4OTE2MDItNWRhNjEzYzAtZjllMS00YjJmLWJmNGUtODVkMjhjNzIwYmM4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzExVDEzMDUzOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTI1ZDNlMDNlMWEwNzIyZTdlYThiMTA2YzAxNDU0NWRhNDc5NzQ1Y2RjYWVkZGI0YTE3ZTBiYWFhMWRjM2M0MzgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0._nlegUqpqbqCblEZU4MI9Xsd4n8wffMye7LuY44jNFg) 同样遇到了，有解决吗

Qwen2-72B-Instruct-gptq-int4重复问题

已收到您的信件！

Some dependency versions and quick inference

ok, thanks so much

Some dependency versions and quick inference

hi, sorry for bothering you. We still haven't seen the correct links, thank you!

Some dependency versions and quick inference

get it! thank you so much!

Some dependency versions and quick inference

已收到您的信件！

多卡推理

已收到您的信件！

训练PPO出现问题：ValueError: Target module ModuleDict( (default): Identity() (reward): Identity() ) is not supported. Currently, only the following modules are supported: `torch.nn.Linear`, `torch.nn.Embedding`, `torch.nn.Conv2d`, `transformers.pytorch_utils.Conv1D`.

In 'trainer_utils.py' file, I replaced the loading of adapter weights with directly loading the merged model. Then, I modified it to use the path of the merged model. Then it...