Baichuan-7B icon indicating copy to clipboard operation
Baichuan-7B copied to clipboard

[Question] pretrain 的训练 config

Open cicirori opened this issue 1 year ago • 0 comments

Required prerequisites

Questions

基于上述的几个优化技术,我们在千卡A800机器上达到了7B模型182Tflops的吞吐,GPU峰值算力利用率高达58.3% 。

首先感谢贵司的开源精神~

然后就是这里达到对应吞吐的时候的 pretrain config 可以放出来吗~

Checklist

  • [X] I have provided all relevant and necessary information above.
  • [X] I have chosen a suitable title for this issue.

cicirori avatar Jun 15 '23 12:06 cicirori