TonyUSTC

icon indicating an email [email protected]

icon company baidu icon location beijing

Results 4 comments of TonyUSTC

[Question] Use api

请问解决了吗？遇到同样的困惑。

为什么我总是训练到一半告诉我内存不足，

> > A10的显存为24GB，全量参数训练7B模型还是略显不足。我这边使用V100全量参数训练7B模型也很吃力。 > > 如果是进行指令微调，结合你的训练资源情况，建议使用QLoRA，也能取得很不错的效果。 > > 我使用QLoRA进行微调时报错如下，我在网上找了很久都没解决 > > ``` > Warning: import flash_attn fail, please install FlashAttention to get higher efficiency https://github.com/Dao-AILab/flash-attention > Loading checkpoint shards: 100%|████████████████████████|...

RuntimeError: FlashAttention only support fp16 and bf16 data type

请问这个问题解决了吗

[Bug]: initializing multiple LLM classes simultaneously on the same GPU get an error

> There should be no need to use different `LLM` instances on the same GPU because even when the model is small, we can utilize the GPU for other optimizations...