Fine-tune REALM on a different set of documents for retrieval

[mchari]

Hi, I was wondering if there is a way to take the pre-trained REALM model checkpoints and fine tune them on another much smaller set of documents ? As a first step, it would suffice to just fine tune the ICT retriever.

Thanks!

[detaos]

I'm wondering the same thing, but for a slightly different use-case. I'm wondering how to add/update/remove documents over time without pretraining from scratch each time.