how to load the fie tuned model?

[sreebalaji2418]

Hi @NielsRogge I have fine tuned the trOCR model and saved in a directory. how to load the model and infer it can u pls help me

[dhea1323]

I have the same problem. I've already fine tuned and saved it. Then, load the model and generate it to an image, but i got an error.

Here's the script,

#Load the image
image = Image.open('path/to/image.jpg')
pixel_values = processor(image, return_tensors="pt").pixel_values
print(pixel_values.shape)
#Load the model
model = VisionEncoderDecoderModel.from_pretrained("path/to/model/")
#Generate the image
generated_ids = model.generate(input_ids=pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(generated_text)

The following errors occurs,

ValueError: `bos_token_id` has to be defined when no `input_ids` are provided.

[NielsRogge]

Hi,

I'll have to update my notebook, because now you don't need to do input_ids=pixel_values anymore. You can just do model.generate(pixel_values).

[dhea1323]

Great ! It works, thank you so much

[sreebalaji2418]

Thanks

[arain60gb]

I face this problem kindly guide me how reslove this problem.Thanks @NielsRogge @dhea1323

image = Image.open('/content/drive/MyDrive/data/test_img/p06-030-s01-00.png') pixel_values = processor(image, return_tensors="pt").pixel_values print(pixel_values.shape)

ValueError: operands could not be broadcast together with shapes (384,384) (3,)

[mpiricsoftware]

Hey Guys! Just confirming.

from transformers import TrOCRProcessor
processor = TrOCRProcessor.from_pretrained("microsoft/trocr-base-handwritten")
pixel_values = processor(image, return_tensors="pt").pixel_values

We are assigning processor instance with pretrained model only. Is it correct?

and for model we are using saved model directory.

from transformers import VisionEncoderDecoderModel
model = VisionEncoderDecoderModel.from_pretrained("/content/drive/MyDrive/custom_model/")