ShareLer

[Question] There is a difference in the logic of Figure 3 and Equation 22 at the part of MTP in the technical report.

1

After reading and deriving the formula for the MTP part, I found that there is a difference in the logic of Equation 22 and Figure 3. Since the subscript [1:T−k]...

fix adapter in/out_features size for share_expert when use overlap.

2

> [!IMPORTANT] > The `Update branch` button must only be pressed in very rare occassions. > An outdated branch is never blocking the merge of a PR. > Please reach...

### Checklist Before Starting - [x] Search for similar PR(s). ### What does this PR do? Fix megatron model merger. ### High-Level Design > Demonstrate the high-level design if this...