CLIP icon indicating copy to clipboard operation
CLIP copied to clipboard

Published 20 hours ago • verified publisher icon openai

How to train CLIP from scratch？

Open newbietuan opened this issue 2 years ago • 1 comments

Hi, Thanks for the great work. Due to the needs of specific tasks, i want to train CLIP from scratch without using BPE coding and the length limit of 77, how should i do? about length_limit, it seems that is ok not to use simple_tokenizer.py, so the mainly problem is how to train CLIP from scratch？ Very thanks.

newbietuan avatar Jul 20 '22 16:07 newbietuan

It seems that OpenAi doesn't provide a code that can train CLIP from scratch. You could refer to the codebase.https://github.com/mlfoundations/open_clip Hope that is helpful for you.

jinghaoliu avatar Jul 25 '22 06:07 jinghaoliu