NielsRogge

MarkupLM is now part of the Transformers library, feel free to close this issue :) * Docs: https://huggingface.co/docs/transformers/model_doc/markuplm * Demo notebooks: https://github.com/NielsRogge/Transformers-Tutorials/tree/master/MarkupLM

No but it's basically the same as [this notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/TrOCR/Fine_tune_TrOCR_on_IAM_Handwriting_Database_using_Seq2SeqTrainer.ipynb), except that you need to swap your model with a PEFT model. In code: ``` from transformers import VisionEncoderDecoderModel from peft...

Hi, regarding which processor to use, it doesn't matter, they both use the same vocabulary and image preprocessing settings. For hyperparameter optimization, I would definitely experiment with number of epochs,...

That's an interesting, weird issue. Could NMS (non-maximum suppression) help here?

Hi, Batched inference is supported by default in the Transformers library as shown [here](https://huggingface.co/docs/transformers/main/en/model_doc/table-transformer#transformers.TableTransformerForObjectDetection.forward.example). Just pass multiple images to the image processor, and multiple target sizes to the `post_process_object_detection` method.

Awesome! I see all models are Keras models. Are you leveraging `from_pretrained_keras` to load models directly from the hub? https://huggingface.co/docs/hub/keras

Hi, Batched inference with LLaVa is supported in Hugging Face Transformers. See here for an example: https://github.com/huggingface/transformers/blob/a49f4acab3c1eea82907e12f82eafbd4673deb39/tests/models/llava/test_modeling_llava.py#L245.

Yes I'm aware of that, this is being addressed in https://github.com/huggingface/transformers/pull/29850 It will be part of the next Transformers release!