Phil Wang

tremendous! :pray: thank you for validating so quickly!

> For B/32 I'm seeing same accuracy after fine tune with no dropout https://wandb.ai/rom1504/open-clip/reports/B-32-patch-dropout-experiments--VmlldzozMDgxODcx > > And speed up for large model + large batch size > > Probably worth...

> Oh yeah and maybe add a small section in readme which section were you thinking?

@rom1504 i made a guess to what you wanted https://github.com/mlfoundations/open_clip/pull/272/commits/6506fcb0a7d71a940ee4b10d2ab0e84b76fec747

@ronghanghu @haooooooqi thank you for your great paper! it is so rare when something actually works :smile:

@calclavia I'm really interested in this as well! Phil had told me that it wasn't possible through email, but maybe I didn't explain it correctly

@calclavia yes, me too... i feel like the whole "linear" attention line of research is being held back by this

@calclavia have you tried cuda-python yet?

yea i can take this on, maybe when i have some spare time next weekend and can get into software engineering mode

ahh ok, yeah, I'll let you all decide what is best it is low priority for sure