donut
donut copied to clipboard
clovaai
Can the model be finetuned with 8 GB cuda memory?
Thank you for this incredible model! As the title says, I have an 8 GB gpu, so is there a way to finetune the model locally? I have tried decreasing the input_shape to 1280x960. I assume this is the lowest value I can decrease it to as I read from the other issues. I set the train_batch_size to 1, too. So, would anyone provide me with more suggestions to fit my constraints? Thanks a ton.
There is no offical code snipped in donut to facilitate the same Try incorporating the suggestions in the link here https://spell.ml/blog/gradient-checkpointing-pytorch-YGypLBAAACEAefHs
Let us know if it works
@inesriahi reducing either of these parameters will help here: input_size: [2560, 1920] ---> [1920, 1280] or [1280, 960] or some lower value (you need to find a lower value where you are not getting any errors like 720, 960, 1080) max_length: 128 ---> 64 or 100
@inesriahi reducing either of these parameters will help here: input_size: [2560, 1920] ---> [1920, 1280] or [1280, 960] or some lower value (you need to find a lower value where you are not getting any errors like 720, 960, 1080) max_length: 128 ---> 64 or 100
Is there a lower input size value that doesn't produce errors?
960x640 works, i guess that the input dimensions should be a multiple of 320
Going below this resolution might give junk, as the pdf might become pixelated and fonts not visible
@inesriahi i'm facing issues running a simple inference of the model on an 8GB GPU. do you have a code snippet that does put the model to the device and does a simple inference please ?
I was not able to run donut on 8GB VRAM, i think minimum is 12 GB and that using smaller input_size, on a 16GB card i get 80-90 % vram usage using input_size: [960, 960]
@Wyzix33 fixed ! The problem was Cudnn not being installed . It can run on 8gb vram ! I'm talking about inference of course, not fine tuning
