feed_forward_vqgan_clip icon indicating copy to clipboard operation
feed_forward_vqgan_clip copied to clipboard

Published 20 hours ago • verified publisher icon mehdidc

Finetuing CLIP to improve domain-specific performance

Open afiaka87 opened this issue 2 years ago • 1 comments

It's quite easy to finetune one of the Open AI CLIP checkpoints with this codebase:

https://github.com/Zasder3/train-CLIP-FT

Uses pytorch-lightning. May be worth pursuing

afiaka87 avatar Aug 02 '21 13:08 afiaka87

I would also be curious to see if training/fine-tuning CLIP on a higher resolution (e.g. 512x512 instead of 224) would also lead to better image quality for higher resolutions( >= 512)

mehdidc avatar Nov 05 '21 09:11 mehdidc