AttributeError: 'BertModel' object has no attribute 'attn_implementation'

[pinnareet]

System Info

Google Colaboratory

• transformers version: 4.41.0
• Platform: Linux-6.1.85+-x86_64-with-glibc2.35
• Python version: 3.10.12
• Huggingface_hub version: 0.23.0
• Safetensors version: 0.4.3
• Accelerate version: not installed
• Accelerate config: not found
• PyTorch version (GPU?): 2.3.0+cu121 (True)
• Tensorflow version (GPU?): 2.15.0 (True)
• Flax version (CPU?/GPU?/TPU?): 0.8.3 (gpu)
• Jax version: 0.4.26
• JaxLib version: 0.4.26
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: No

Who can help?

No response

Information

• [ ] The official example scripts
• [X] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [X] My own task or dataset (give details below)

Reproduction

https://colab.research.google.com/drive/1ncTM6lk6ZOfnwSFr_jaXL2zwwvWA-JRj?usp=sharing

Expected behavior

It is not supposed to have an error.

[amyeroberts]

Hi @pinnareet, thanks for raising this issue!

From the linked colab, it looks the error is coming from the bertopic library and the model implementation there. I'd suggest opening an issue in the repo

[aymenkrifa]

@amyeroberts, same problem here with only using Transformers (4.41.0)

with torch.no_grad(): outputs = model( input_ids, attention_mask=attention_mask, token_type_ids=None )

I'm using Bert with Torch version 2.2.1

[amyeroberts]

@aymenkrifa Could you share how the model is created and your running env (The running environment: run transformers-cli env in the terminal and copy-paste the output) ?

[b5y]

Also had this issue when tried to load model using sentence-transformers==2.3.1 with transformers==4.41. After changing the version transformers==4.37 and setting sentence-transformers==2.7.0 the problem is gone.

Not sure if downgrading transformers version could help for everyone, but in my case it fixed the issue.

[amyeroberts]

@b5y Thanks for sharing! It seems this might be a compatibility issue then between transformers and sentence transformers cc @tomaarsen

[b5y]

@amyeroberts I don't think there is any compability issue here. The error itself came from transformers library and it duplicates the error shown in this original post.

UPDATE: it was fine for more than 1 month, just came out recently on Wednesday.

[amyeroberts]

@b5y Could you share a code snippet which reproduces this and full error traceback?

[GeorgWa]

We see the same issue with transformers-4.41.1:

0:00:00.410061 INFO: Traceback (most recent call last):
  File "alphadia/cli.py", line 333, in run
    plan = Plan(
           ^^^^^
  File "alphadia/planning.py", line 126, in __init__
    self.load_library()
  File "alphadia/planning.py", line 229, in load_library
    spectral_library = pept_deep_prediction(spectral_library)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "alphadia/libtransform.py", line 40, in __call__
    return self.forward(*args)
           ^^^^^^^^^^^^^^^^^^^
  File "alphadia/libtransform.py", line 306, in forward
    model_mgr = ModelManager(device=device)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "peptdeep/pretrained_models.py", line 300, in __init__
    self.ms2_model:pDeepModel = pDeepModel(mask_modloss=mask_modloss, device=device)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "peptdeep/model/ms2.py", line 385, in __init__
    self.build(
  File "peptdeep/model/model_interface.py", line 248, in build
    self.model = model_class(**kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^
  File "peptdeep/model/ms2.py", line 171, in __init__
    self.hidden_nn = building_block.Hidden_HFace_Transformer(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "peptdeep/model/building_block.py", line 224, in __init__
    self.bert = BertEncoder(self.config)
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers/models/bert/modeling_bert.py", line 643, in __init__
    self.layer = nn.ModuleList([BertLayer(config) for _ in range(config.num_hidden_layers)])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers/models/bert/modeling_bert.py", line 643, in <listcomp>
    self.layer = nn.ModuleList([BertLayer(config) for _ in range(config.num_hidden_layers)])
                                ^^^^^^^^^^^^^^^^^
  File "transformers/models/bert/modeling_bert.py", line 558, in __init__
    self.attention = BertAttention(config)
                     ^^^^^^^^^^^^^^^^^^^^^
  File "transformers/models/bert/modeling_bert.py", line 476, in __init__
    self.self = BERT_SELF_ATTENTION_CLASSES[config._attn_implementation](
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: '_Pseudo_Bert_Config' object has no attribute '_attn_implementation'

0:00:00.410134 ERROR: '_Pseudo_Bert_Config' object has no attribute '_attn_implementation'

[amyeroberts]

@GeorgWa Could you share a reproducible code snippet? From the traceback, it looks like a non-transformers config file is being used (we don't have a _Pseudo_Bert_Config class)

[GeorgWa]

Yes, will do

[GeorgWa]

Sorry for the confusion, it looks like adding _attn_implementation to the config object solves the issue on our side. I'm still unsure why we have to add a private config field, Will check with my colleauge as I'm not an expert in this part of the codebase.

I would expect _Pseudo_Bert_Config should be inherit the default config or maybe a check if the provided config is an instance.

Thank you!

from transformers.models.bert.modeling_bert import BertEncoder

class _Pseudo_Bert_Config:
    def __init__(self, 
        hidden_dim=256, 
        intermediate_size=1024,
        num_attention_heads=8,
        num_bert_layers=4,
        dropout=0.1,
        output_attentions=False,
    ):
        self.add_cross_attention = False
        self.chunk_size_feed_forward = 0
        self.is_decoder = False
        self.seq_len_dim = 1
        self.training = False
        self.hidden_act = "gelu"
        self.hidden_dropout_prob = dropout
        self.attention_probs_dropout_prob = dropout
        self.hidden_size = hidden_dim
        self.initializer_range = 0.02
        self.intermediate_size = intermediate_size
        self.layer_norm_eps = 1e-8
        self.num_attention_heads = num_attention_heads
        self.num_hidden_layers = num_bert_layers
        self.output_attentions = output_attentions
        #self._attn_implementation = "eager" uncommenting solves the issue

bert = BertEncoder(_Pseudo_Bert_Config())

[amyeroberts]

@GeorgWa It's assumed the config passed to the transformers model is a child of PretrainedConfig. When this config is created (either directly or through AutoModel calls) then the private attribute _attn_implementation will be set

[liqi6811]

I got the same error when working in Databricks for model serving, the code works well locally, but when create a endpoint, I got this error:

"error_code": "BAD_REQUEST", "message": "Encountered an unexpected error while evaluating the model. Verify that the input is compatible with the model for inference. Error ''BertModel' object has no attribute 'attn_implementation''"

The same code I deployed a few weeks ago, it worked fine. I tried to downgrade transformers version, it did not work.

any tips? Thanks!

[amyeroberts]

Hi @liqi6811, could you a reproducible code snippet? In particular, one which can run locally but fails when deploying on databricks?

[liqi6811]

@amyeroberts I worked it out by specifying transformers and sentence-transformers version when log model.

# Start an MLflow run and log the model
with mlflow.start_run():
    mlflow.pyfunc.log_model("model", 
                            python_model=wrapped_model, 
                            input_example=df_sample, 
                            signature=signature,
                            pip_requirements=["transformers==4.37", 
                                              "sentence-transformers==2.7.0", 
                                              "mlflow==2.5.0"])

[amyeroberts]

@liqi6811 Thanks for clarifying cc @tomaarsen re sentence-transformers <-> transformers compatibility

[Wmuntean]

Experiencing the same issue with transformers==4.40.2 and sentence_transformers==3.0.1.

Anyone figure out a workaround?

EDIT: Looks like upgrading to transformers==4.41.2 solved the issue for me.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[ECMGit]

I solved on my end when I downgrade transformers from 4.42.3 --> 4.40.2 my other related libraries version

sentence-transformers     3.0.1
transformer-engine        1.8.0+37280ec

[trantrikien239]

Hi, I'm still having this issue without using SentenceTransformer. Would love to re-activate it. Thanks