Mayank Mishra
Mayank Mishra
@stas00, I believe the code is in good shape. You can start reviewing. Also, I'll start updating the README for instructions tomorrow
1. I found it easier to deploy using DeepSpeed-MII and leverage that for CLI. But I wan't really sure of the overhead it causes, so still using the barebones DS-inference...
Also, I am not sure if this works for other models (a few strings are hardcoded). I think for now we should stick to BLOOM. I can open up another...
> you don't need to broadcast anything for non-server usage. It sounds like you built `cli.py` as a client to the server, but that's too complicated for someone who just...
So, just to summarize for cli.py: You want the option for user to provide an input text file?
Also, @stas00 , I am fine with both. If we want, we can keep this branch as a separate branch if users want to leverage a standalone system and maybe...
Thanks for the comments Stas. Will incorporate the changes I have added back the older scripts in scripts/inference folder (original path). Will update README.md
> > I have added back the older scripts in scripts/inference folder (original path). > > and let's move the 2 solutions to their respective sub-dirs: > > ``` >...
Done. Added README too @stas00
I want to remove the temporary json created while caching. However, the new commit in DeepSpeed master branch is not working for me. https://github.com/microsoft/DeepSpeed/pull/2132#issuecomment-1221592051 Waiting for a fix @RezaYazdaniAminabadi