v100 lora fp16训练后有什么推理加速的方法吗

[ZTurboX]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

v100 lora fp16训练后有什么推理加速的方法吗

Expected Behavior

No response

Steps To Reproduce

v100 lora fp16训练后有什么推理加速的方法吗

Environment

- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :

Anything else?

No response

[nghuyong]

+1 希望官方能出一个lora微调后加速推理的方案

[datalee]

+1 同求