zhangfan-algo

想问下如何设置自定义的系统提示词呢

3

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 哪个参数可以设置呢 ### Expected behavior _No response_ ### System Info _No response_ ### Others _No...

使用BAdam多节点训练报错

1

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction 报错信息：raise ValueError("Layer-wise BAdam does not yet support distributed training, use ratio-wise BAdam.") ### Expected...

目前测试数据上下文比较长，且无法缩短。请教一下有没有方法可以支持一下更长的上下文 

import flash attention errror

4

env :cuda 12.3 pytorch 2.2.2 Failed to import transformers.models.qwen2.modeling_qwen2 because of the following error (look up to see its traceback): /mnt/pfs/zhangfan/system/anaconda/envs/swift/lib/python3.10/site-packages/flash_attn-2.5.5-py3.10-linux-x86_64.egg/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: ZN2at4_ops15sum_IntList_out4callERKNS_6TensorEN3c1016OptionalArrayRefIlEEbSt8optionalINS5_10ScalarTypeEERS2 RuntimeError raise RuntimeError(: Failed to import...

官方可以提供一份run train_with_qlora.py的示例脚本吗

4

主要是不太懂需要配置那些参数,辛苦大佬帮忙给一份示例跑train_with_qlora微调代码的脚本

跑tigerbot-llama2-70b-chat,环境的版本是正确,但是代码会报错

2

 

模型能力评测和数据

1

想看下模型各项能力如何,例如视觉推理能力,OCR能力测评.以及预训练数据集和指令微调数据的训练代码以及数据可以开源一下吗 包括预训练和指令精调的

Can we support more languages?

1

like Chinese and more.