ruanslv

Isn't this similar to what Binh is working on? Creating a consolidated sharding script https://github.com/facebookresearch/metaseq/issues/376

Is it possible to run $ git mv so that the new script becomes a second version of the original one? Makes it easier to review. Also lint failed, maybe...

I'm rewriting the model worker, to make it work better with FastAPI front-end and more generic (accept more models). Was thinking of doing the whole work in metaseq-internal, until we...

> I think that merging this (plus a little more I want to add) will make it really easy for anyone to start developing with the API But my proposal...

I'm rewriting interactive_hosted to be Pydantic (https://pydantic-docs.helpmanual.io/ -- i.e. use explicitly defined Request/Response classes as opposed to JSON), so most of it will be thrown away (we don't want to...

Doesn't look like merge with latest main branch worked, I still see conflicting files. If you want some help on this I'd need write access to your fork

Our inference code shows a way to add tokens to the right, not to the left. I believe that's a common way to do padding: https://github.com/facebookresearch/llama/blob/556949fdfb72da27c2f4a40b7f0e4cf0b8153a28/llama/generation.py#L167-L170