AdvancedLiterateMachinery
AdvancedLiterateMachinery copied to clipboard
Increase max_seq_length
How can we pass more tokens than max_seq_length. I have more tokens than 512
You can split the document into several sub-documents to fit the length restriction of the model.
You can split the document into several sub-documents to fit the length restriction of the model.
Yes, we can certainly proceed with this approach, but it's important to note that it may result in missing out on some key value pairings. For example, if the image document represents a table, using this method might lead to the omission of crucial information that could be vital for accurate understanding and analysis. It is essential to consider the potential drawbacks and limitations while employing this approach.