Running NBT on a custom dataset

[azouaoui-cv]

Hi Jiansen, Thanks for the comprehensive repository you set up!

I'm in the process of trying to use NBT on a custom annotated dataset. To my understanding, this is not possible right out of the bat (see #10) as there has been no commit since this issue was closed that enables using a custom image.

I was wondering; if you were to implement this possibility, how would you proceed? What are the main changes you foresee? Is it dealing with the vocabulary? the custom objects detectors ?

My idea would be to start by considering the custom images as being part of the datasets the models got trained on (COCO or Flickr30k), hence we could use the vocabulary and object detectors models from these datasets to generate captions on our annotated dataset. Any thoughts?