Andrew Gu

https://github.com/Tiiiger/QPyTorch/blob/176bf2a4b02021b116991d9b2f4d920861725f26/qtorch/number.py#L23 Should this be `[-2^{wl-fl-1}+2^{-fl}, 2^{wl-fl-1}-2^{-fl}]`? This seems to be how symmetric fixed-point is used in the [WAGE paper](https://arxiv.org/pdf/1802.04680.pdf) (bottom of page 3). For the current formula, if `wl=8` and...

[FSDP] Add back `forward_prefetch`

16

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #85177 * #85176 - This implements explicit forward prefetching following the static 1st iteration's pre-forward order when `forward_prefetch=True` in the FSDP constructor....

[FSDP] Simplify backward prefetch implementation

6

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #85177 * __->__ #85176

[FSDP][BE] Better error msg for incorrect device for training

1

Stack from [ghstack](https://github.com/ezyang/ghstack): * #92035 [FSDP][RFC] Enforce rank `r`'s current device is `cuda:r` * #91767 [FSDP] Do not clean FQNs even for `use_orig_params=True` * #92031 [FSDP][BE] Improve `device_id` + CPU...

[FSDP] Do not clean FQNs even for `use_orig_params=True`

6

Stack from [ghstack](https://github.com/ezyang/ghstack): * #92035 [FSDP][RFC] Enforce rank `r`'s current device is `cuda:r` * **#91767 [FSDP] Do not clean FQNs even for `use_orig_params=True`** * #92031 [FSDP][BE] Improve `device_id` + CPU...

[Lint] Add FSDP/composable API files to ufmt include

2

Stack from [ghstack](https://github.com/ezyang/ghstack): * **#90873 [Lint] Add FSDP/composable API files to ufmt include** This PR adds FSDP and composable API files to `.lintrunner.toml` so that (1) lintrunner enforces that those...

[FSDP][BE] Rename `prefixed_param_names` -> `fqns` for consolidation

1

Stack from [ghstack](https://github.com/ezyang/ghstack): * #92035 [FSDP][RFC] Enforce rank `r`'s current device is `cuda:r` * #91767 [FSDP] Do not clean FQNs even for `use_orig_params=True` * #92031 [FSDP][BE] Improve `device_id` + CPU...

[FSDP][RFC] Enforce rank `r`'s current device is `cuda:r`

2

Stack from [ghstack](https://github.com/ezyang/ghstack): * **#92035 [FSDP][RFC] Enforce rank `r`'s current device is `cuda:r`** * #91767 [FSDP] Do not clean FQNs even for `use_orig_params=True` * #92031 [FSDP][BE] Improve `device_id` + CPU...

[FSDP][BE] Improve `device_id` + CPU offload test

1

Stack from [ghstack](https://github.com/ezyang/ghstack): * #92035 [FSDP][RFC] Enforce rank `r`'s current device is `cuda:r` * #91767 [FSDP] Do not clean FQNs even for `use_orig_params=True` * **#92031 [FSDP][BE] Improve `device_id` + CPU...

[Reland][FSDP] Do not clean FQNs for `use_orig_params=True`

1

Stack from [ghstack](https://github.com/ezyang/ghstack): * **#92662 [Reland][FSDP] Do not clean FQNs for `use_orig_params=True`** The last PR (https://github.com/pytorch/pytorch/pull/91767/) had a land race and got reverted. This is a re-land. cc @mlazos @soumith...