ChengShuting

关系抽取 (Relation Extraction, RE)模型不支持转onnx格式

1

**  **

repeated answer:When I use vllm with Qwen-7b-chat the generated text is x lnot end until the maength, with the repeated answer

1

sampling_parameters = { "temperature": "0", "top_p": "0.5", "max_tokens": "300"} python3 client.py 

Why did I use llama2-7B when pruningthe model to the same size as the original model

1

Why did I use llama2-7B when pruningthe model to the same size as the original model

How to solve the problem of errors when loading qwen1.5-7B (using two GPUs) and llama3-8B (using two GPUs) models simultaneously using tritonserver?

3

### System Info env: NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 V100 16G*8 docker images: nvcr.io/nvidia/tritonserver:24.02-trtllm-python-py3 ### Who can help? _No response_ ### Information - [ ] The official...