larekrow comments

Repositories
Issues
Comments

Results 5 comments of


                                            larekrow

[BUG] deepspeed.init_inference() erroneously attempts to copy out of meta tensor

Hi, any updates?

[BUG] deepspeed.init_inference() erroneously attempts to copy out of meta tensor

Hi @qtli, appreciate the suggestion, but I did not use `'replace_method': 'auto'` following PR #2831. I did try to run it again upon your suggestion for good measure though --...

Will OPT-IML-175B be supported?

I am looking for the conversion scripts/steps for the OPT-IML-175B weights, similar to the [conversion for the OPT-175B weights](https://github.com/alpa-projects/alpa/tree/main/examples/llm_serving#convert-opt-175b-weights-into-alpa-formats). The metaseq OPT-175B weights are given as 992 FSDP shards while...

Deprecate convert_to_singleton

`reshard_mp.py --num-output-parts 1` currently does not work with the OPT weights. Please see #695.