ethanc8

GAIR [has finetuned](https://github.com/GAIR-NLP/anole) Chameleon-7B to support image generation. ([weights on HuggingFace, in PyTorch format](https://huggingface.co/GAIR/Anole-7b-v0.1), [inference and finetuning code](https://github.com/GAIR-NLP/anole)). Will this PR's code be able to support Chameleon models that have...

> It would be best to, instead of renaming them, do a similar version ifdef for the version of macOS this change was made in. We always want to be...

> Could you please add the #ifdef that Gregory requested so we can merge this. I will do that tomorrow.

I have rebased it on `master` and will try updating the patches soon.

@fredkiefer @gcasa I believe this is ready to merge into master. The CI errors are due to unrelated issues with the workflow YAML definitions.

I have [llamafied InternLM2.5-7B](https://huggingface.co/ethanc8/internlm2_5-7b-llamafied), and [tried to open it in Unsloth](https://colab.research.google.com/drive/1bpUQNwSJxFsNTPSfD1Mjr9LLpFevy0cO#scrollTo=QmUBVEnvCDJv). I get ``` /usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py in LlamaAttention__init__(self, config, layer_idx) ValueError: Unknown RoPE scaling type dynamic ```

This model has ```json "rope_scaling": { "factor": 2.0, "type": "dynamic" }, ``` in [its config.json](https://huggingface.co/ethanc8/internlm2_5-7b-llamafied/blob/main/config.json)

Open LLM Leaderboard also seems to be having trouble with its dynamic rope_scaling: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/862

In the other closed issue, you mentioned that RoPE scaling can be disabled in order to finetune the model. I will try that.

I actually have no idea, will probably need to read the internlm remote code: https://huggingface.co/internlm/internlm2_5-7b/blob/main/modeling_internlm2.py