[Feat] Heterogeneous Code Part 1: Add Model and Module Code for Chameleon Lumina

[zhhsplendid]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

This PR and next PR series will be the code adding heterogeneous support and Chameleon model for InternEvo.

We plan to merge those PRs:

1. Adding Chameleon model code.
2. Adding Chameleon DataLoader code.
3. Adding some training code for Chameleon, for example, z_loss, discard large grad norm, etc.
4. Configuration of Chameleon and some tests for integrating above.
5. Heterogeneous support, which is flag-controlled cpu+gloo p2p communication and unbalanced pipeline parallelism.
6. More tests if needed.

This PR is the first one: adding Chameleon Model code.

Modification

As described above, this PR is the first of a series of PRs: Adding Chameleon model code.

BC-breaking (Optional)

None

Use cases (Optional)

We will add use case after the configuration PR so that we can show the training use case.

Checklist

Before PR:

• [x] Pre-commit or other linting tools are used to fix the potential lint issues.
• [x] Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
• [x] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• [x] The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

• [x] If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects.
• [ ] CLA has been signed and all committers have signed the CLA in this PR.