[BUG] Cannot Initialize Any Models in transformers==4.41.2

[JiahuiKChen]

Expected Behavior

When using the given model initialization code:

from open_flamingo import create_model_and_transforms

model, image_processor, tokenizer = create_model_and_transforms(
clip_vision_encoder_path="ViT-L-14",
clip_vision_encoder_pretrained="openai",
lang_encoder_path="anas-awadalla/mpt-7b",
tokenizer_path="anas-awadalla/mpt-7b",
cross_attn_every_n_layers=4
)

the model should be initialized

Current Behavior

Failure with error: ValueError: We require the attribute name for the nn.ModuleList in the decoder storing the transformer block layers. Please supply this string manually. Full error output here: https://gist.github.com/JiahuiKChen/d3695f31bcf7702bdf0c4ea86a57a77b

Steps to Reproduce

Run any of the create_model_and_transforms code snippets.

I'm getting errors for all the models. Different error for 4B models, see report here.

Environment

Python 3.12.3
transformers 4.41.2
torch 2.3.1

Detailed Description

No changes made, no code other than the import and initialization method

[chenyil6]

I use pip install transformers==4.30.2 and then it worked.