DeepSpeed-MII
DeepSpeed-MII copied to clipboard
Expose DS-inference and ZeRO-inference configs to user
trafficstars
After #25 is complete we want to expose all DS-inference configs (https://deepspeed.readthedocs.io/en/latest/inference-init.html#deepspeed.init_inference) and ZeRO inference configs in the MII config dictionary.
ZeRO inference currently takes the general deepspeed config json path. Do we want to expose DS-inference config in the deepspeed config file as well?
Good point @cli99, I forgot that the zero-inference PR adds ds_config as a MII deploy argument. Let's leave it like that for now, but maybe we can merge some of these configs in the near future.