Mitchell Wortsman
Results
15
issues of
Mitchell Wortsman
Used the implementation from https://github.com/lucidrains/lion-pytorch (thanks @lucidrains) Paper https://arxiv.org/abs/2302.06675
int8 inference
Hello, I am using `torch.cuda.amp.autocast` with `bfloat16`. I noticed that the xformers `RotaryEmbedding` produces `float32` outputs, which then requires casting before passing to `memory_efficient_attention`. However, this raises the question --...
bug