Tingfeng Lan
Tingfeng Lan
""str1"\n"str2"",单独字符串要引号，最外围一个银行，以\n分割
Additional information for your reference: https://github.com/hiyouga/LLaMA-Factory#distributed-training.
Repo link for ChatGLM : https://github.com/THUDM/ChatGLM-6B
Two ways to solve this problem. - A better batch update strategy. - A better threshold to utilize more vmem.
> We can support the `Trainer` in lighting and implement a lighting [callback](https://github.com/Lightning-AI/lightning/blob/master/src/lightning/pytorch/callbacks/callback.py) Thank you for your prompt and helpful response! I'll definitely look into implementing the lighting callback as...
Repo link for Stanford alpaca : https://github.com/tatsu-lab/stanford_alpaca
However, there are mystical problems mentioned by some [blogs](https://zhuanlan.zhihu.com/p/651218251). That is to say, we may experience unexpected slow down by setting an even number batch size.
@delock @sfc-gh-truwase @tohtana, I've moved the discussion to this issue, feel free to continue the conversation here.
@delock Thanks a lot for the quick and thoughtful feedback! This benchmark looks great — I’ll incorporate the results and also refine the ZenFlow benchmarks and CPU-binding part accordingly. I’ve...
@delock Give me some time to check this part in more detail, I’ll get back with specifics. Could you share the exact command you used to run this? I’d like...