Transformers-Tutorials
Transformers-Tutorials copied to clipboard
NielsRogge
Clarification on the Nougat Transformers
@NielsRogge, thanks for the tutorials, I am particularly interested in the Nougat one and have a question.
Nougat can provide the text extraction from the pdfs, I was wondering, can it extract Tables/structured data and Images too? Have you tried this. By images, I don't mean pdfs as images but rather the images within the pdfs.
My end goal is to store the extracted data separately into 3 sections, 1. Text 2. Images 3. Tables Would love to hear on this matter from you.
Yes, you can train a Nougat/Donut model which takes in images of tables and generates the corresponding content in key-value pairs. You just need a high quality dataset of (table image, table content) pairs.
Yes, you can train a Nougat/Donut model which takes in images of tables and generates the corresponding content in key-value pairs. You just need a high quality dataset of (table image, table content) pairs.
I mean, Nougat was specifically designed for academic papers right? So it should have been altrady tained to extract "Structured Data" since any research paper will contain Tables and text together??
Yes you could fine-tune Nougat on additional data, you could benefit from Nougat's pre-training.
