Baichuan-7B
Baichuan-7B copied to clipboard
[Question] pretrain 的训练 config
Required prerequisites
- [X] I have read the documentation https://github.com/baichuan-inc/baichuan-7B/blob/HEAD/README.md.
- [X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
- [x] Consider asking first in a Discussion.
Questions
基于上述的几个优化技术,我们在千卡A800机器上达到了7B模型182Tflops的吞吐,GPU峰值算力利用率高达58.3% 。
首先感谢贵司的开源精神~
然后就是这里达到对应吞吐的时候的 pretrain config 可以放出来吗~
Checklist
- [X] I have provided all relevant and necessary information above.
- [X] I have chosen a suitable title for this issue.