f-fuchs

Question regarding Equation 5 Random Mixing in the Paper

1

Should the matrix multiplication not be swapped? $$ RandomMixing(X) = XW_R \Rightarrow RandomMixing(X) = W_RX $$ I think the dimension don`t work for the original equation because you are multiplying...

TimmBackbone.from_pretrained out_indices not working correctly (out_indices is sorted using strings blocks.11 before blocks.3)

3

### System Info - `transformers` version: 4.42.4 - Platform: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35 - Python version: 3.10.14 - Huggingface_hub version: 0.23.5 - Safetensors version: 0.4.3 - Accelerate version: not installed - Accelerate config:...

### Feature request Add a way to remap model weights. ### Motivation At the moment it is not possible to load the weights of a compiled model into the uncompiled...