zhangyu68

> > Hi @zwd003 May you merge the latest main branch and fix the conflicts? Thanks. > > ok hello,I encountered this error when the QPS was increased to 2....

meta_instruction= "你一个名叫ChatLAW，由北京大学团队开发的人工智能助理：\n- 你旨在提供有无害且准确的回答。\n- 你必须拒绝回答非法的问题。\n- 你的回应不能含糊、指责、粗鲁、有争议、离题或防御性。\n- 你的回应必须有礼貌。" ​ query = "公司无故辞退摸鱼员工，是否触犯法律？" # prompt = f"Consult:\n{consult}\nResponse:\n" prompt = f"{meta_instruction}\nConsult:\n{query}\nResponse:\n" inputs = tokenizer(prompt, return_tensors="pt") inputs['input_ids'] = inputs['input_ids'].to(model.device) ​ with torch.no_grad(): generation_output = model.generate(...

遇到了类似问题，lora sft 相同配置下，qwen-14b-chat的GPU利用率能达到90+ moe模型的GPU利用率只有40左右 使用的是llama-factory 训练框架，环境信息如下： ``` nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Tue_Aug_15_22:02:13_PDT_2023 Cuda compilation tools, release 12.2, V12.2.140 Build cuda_12.2.r12.2/compiler.33191640_0 Package...

searching for solutions

how can I export a onnx model by llava-1.5-7b-hf? when I run this command and then get an error: `optimum-cli export onnx --model /workspace/[email protected]/original_models/llava-1.5-7b-hf onnx_model/llava-v1.5-7b --task image-to-text-with-past --trust-remote-code ` `Traceback...

> > I am also getting `ImportError: /lib/python3.8/site-packages/flash_attn_2_cuda.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi` > > I got the same error with PyTorch 2.3.0 when I installed it with the [wheel file](https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.7/flash_attn-2.5.7+cu122torch2.3cxx11abiFALSE-cp311-cp311-linux_x86_64.whl). I...