open_clip [WIP] Add mammut

Initial attempts at adding MaMMuT https://arxiv.org/abs/2303.16839

@mehdidc @JeniaJitsev @rom1504 I don´t know marianna github name can you tag her too?

This is the PR, will add all of you as collaborators.

Sep 19 '23 11:09 gpucce

Initial attempts at adding MaMMuT https://arxiv.org/abs/2303.16839

@mehdidc @JeniaJitsev @rom1504 I don´t know marianna github name can you tag her too?

This is the PR, will add all of you as collaborators.

@marianna13 is the tag. We have a version in https://github.com/SLAMPAI/open_clip_all_at_once , can add you there to work on fusing everything we got on Mammut.

Dec 23 '24 02:12 JeniaJitsev

@JeniaJitsev sure!

Dec 23 '24 06:12 gpucce

@gpucce actually a part from resolving conflicts, the only thing I see that is needed is to support logit_bias for siglip training (e.g., like https://github.com/mlfoundations/open_clip/commit/6f05e1dfe9d19782978466f4a815c44bcf40ad6a#diff-2c6f604038a68c220ab530723839b6716c1ca011291158de70c3f8e17c0a74deR86) and adding missing configs (https://github.com/SLAMPAI/open_clip_all_at_once/commit/0ec7475afb20280eaae6fdfdf389e92edded0cfa). I would say the rest of the modifications are out of the scope of this PR.

Jan 13 '25 08:01 mehdidc

@mehdidc I am not in https://github.com/SLAMPAI/open_clip_all_at_once/

Can you add me there?

Jan 13 '25 08:01 gpucce

I will have a look at fixing conflicts and making this mergeable in the next couple of days

Jan 13 '25 08:01 gpucce

@gpucce Cool ! sure, added you now.

Jan 13 '25 08:01 mehdidc