Train on a custom dataset

[umang13]

Hi, Great work, thanks a lot for your efforts. I was trying to generate images similar to datasets other than IAM/RIMES (for instance Bentham or Washington datasets or maybe some custom datasets). From the top of my head, I could think of two ways,

1. I extract styles for these datasets (or some samples of images from them) and use the pretrained models (maybe the IAM one) provided to generate those images.
2. Retrain the entire model from scratch on these new databases and use that model to generate new images.

Which one would you suggest will work better if at all? Thanks again.

[herobd]

Unless the datasets are very similar to one the model is pre-trained on, you're going to need to train the model. You could try fine-tuning the pre-trained one to make things faster.

[umang13]

Thanks for your response. If I try the fine-tuning approach, do I need to fine tune each component (recognition,encoder and generator) with my data set, or would just fine tuning the generator work as well? Also the code seems to tag data points with authors (most likely to extract author specific styles). If my data set has no information of authors (just images and corresponding ground truth text), can the entire training process be reused or do I need to make significant changes to it?

[herobd]

You probably will need to fine-tune each component. The recognition network will need fine-tuned first, then the encoder and generator will be fine-tuned in the normal training procedure. You could try to just fine-tune the generator, but I think weird things might happend.

You should be able to get by without author information. If it's a single author dataset like Bentham or Washington you can just assign all the instances the same author. If it's a mixed author dataset, you'll just need to set the data_loader -> a_batch_size parameter to 1 (this will prevent multi-instance style extraction). Under both of these scenarios the dataset object can be simplified from mine.