Jane (Yuan) Xu

Add path optimize kwarg to einsum

2

## This PR seeks to: - [x] add c++ support for an optimize path - [x] add python opt_einsum path passthrough - [x] add opt_einsum to OSS requirements, but a...

[adam] Add not torch.jit.is_scripting() as a requirement for switching to fused

3

A "fix" following https://github.com/pytorch/pytorch/pull/90865. Realized that fused is not compatible with torch.jit.is_scripting() when looking at a later line. Took the opportunity to make the code cleaner/slightly more performant (with the...

"multi device" tests get skipped in standard CI

5

## Concern The general concern is that people think they're adding tests to CI but these tests are getting skipped, which is bamboozling. Specifically, I wonder how many tests there...

[nn] zero_grad() set_to_none default True

1

Attempts to fix #92656 Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #92731

[foreach] guard default support on native tensors only

1

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #92730 * __->__ #92923

Make keep-going work for more than linux

3

cc. asked by @zou3519 Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #96974

Thoughts on making the numpy dependency optional?

6

## Proposal Make the numpy dependency optional, if possible. ## Why? Minimizing dependencies is a general goal as it allows a bigger audience to reap the benefits of this library....

Disable torch.backends.opt_einsum to avoid duplicate work

15

## Description Recently, `torch.einsum` has improved to automatically optimize for multi-contractions if the opt_einsum library is installed. This way, torch users can reap benefits easily. However, this change may inadvertently...

In the past 2-3 weeks, these configs have been bouncing up and down. DALLE2_pytorch, Adam, cuda, amsgrad, maximize DALLE2_pytorch, Adam, cuda, default DALLE2_pytorch, Adam, cuda, foreach DALLE2_pytorch, Adam, cuda, fused,...

Add llama_v2_7b_16h to the optim benchmarks once the model supports train

1

This is just a tracking issue to make sure we don't forget cc. @msaroufim