Mitchell Wortsman

Unfortunately we haven't tried that at this time so don't have a good answer for you

I have not tried this but those hyperparameters seem like they should be good, is their any reason to use our checkpoints and not OpenAI's via `--openai-pretrained` ? This may...

Closing for now since it's supported, thank you!

Yes of course!

https://github.com/mlfoundations/open_clip/pull/281

Ya at a high level we were just thinking this would be something beneficial for the community, but no concrete plans -- an open_clip_jax repo would be great!

In the pytorch imagenet example for distributed imagenet eval they have an `aux_val_loader` to handle the case where the test set size is not divisible by num_gpus - do we...

@mehdidc I think this is actually necessary or else you can get different val perf when different numbers of gpus are used, e.g., see this comment: https://github.com/facebookresearch/deit/blob/main/main.py#L221-L223

not sure if it would work but have you by any chance looked at using captions like `"this is a photo of a ','.join(subset)"` where subset iterates over all subsets...

So far at small scale (short B/32 run, batch size 16k), well tuned lion slightly outperforms AdamW (still tuning AdamW). AdamW (LR 2e-3, WD 0.2, betas=0.9, 0.95) = 42.1 Lion...