InternEvo

InternEvo copied to clipboard

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

blankde

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

blankde

when tp size > kv head_number, copy kv head

yingtongxiong

Add docs for 2d-attention

yingtongxiong

## Motivation 1. Fix CI timeout for https://github.com/InternLM/InternEvo/issues/342 **(Completed)** 2. Refine implementation of megatron and mocked dataloader **(Completed)** ## Modification - `internlm/train/pipeline.py` - `internlm/data/*` ## BC-breaking (Optional) None ## Use...

zigzagcai

[QA] loong train 支持packed_sample_into_one=false吗

1

### 描述问题 咨询一下，长文本训练支持样本间的相互隔离吗？

Lzhang-hub

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

li126com

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

blankde

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

blankde

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

li126com