Pix2struct screen2words not working

[lambainsaan]

System Info

• transformers version: 4.27.2
• Platform: macOS-13.2.1-x86_64-i386-64bit
• Python version: 3.10.10
• Huggingface_hub version: 0.13.3
• PyTorch version (GPU?): 2.0.0 (False)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: no
• Using distributed or parallel set-up in script?: no

Who can help?

@younesbelkada

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

When running the code,

from transformers import AutoProcessor, AutoModelForSeq2SeqLM

processor = AutoProcessor.from_pretrained("google/pix2struct-screen2words-large")

model = AutoModelForSeq2SeqLM.from_pretrained("google/pix2struct-screen2words-large")

I am getting the error

[transformers/models/auto/processing_auto.py:270](/codes/pix2struct-tryout/~/codes/pix2struct-tryout/.venv/lib/python3.10/site-packages/transformers/models/auto/processing_auto.py:270), in AutoProcessor.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    267     else:
    268         processor_class = processor_class_from_name(processor_class)
--> 270     return processor_class.from_pretrained(
    271         pretrained_model_name_or_path, trust_remote_code=trust_remote_code, **kwargs
    272     )
    274 # Last try: we use the PROCESSOR_MAPPING.
    275 if type(config) in PROCESSOR_MAPPING:

AttributeError: 'NoneType' object has no attribute 'from_pretrained'

Expected behavior

The model variable must be populated with the model.

[amyeroberts]

Hi @lambainsaan, thanks for raising this issue!

Pix2Struct was merged into main after the 4.27.2 release. To get the most recent version of the codebase, you can install from the dev branch by running: pip install git+https://github.com/huggingface/transformers.

Note: It is not possible to load Pix2Struct with AutoModelForSeq2SeqLM API

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[younesbelkada]

I believe this issue is fixed, closing it now!