vllm Support for MPT-7B and MPT-30B

Support for MPT-7B and MPT-30B

Open mspronesti opened this issue 1 year ago • 4 comments

It would be great if you could support MPT-7B and MPT-30B

Jun 23 '23 08:06 mspronesti

Hi @mspronesti, thanks for brining it up. I believe MPT's architecture is pretty similar to BLOOM. We will add support for both models very soon!

Jun 24 '23 00:06 WoosukKwon

Hi @mspronesti, thanks for brining it up. I believe MPT's architecture is pretty similar to BLOOM. We will add support for both models very soon!

I believe it's also similar to OPT.

Jun 29 '23 15:06 emsi

+1. I think both bloom and mpt just need alibi implementation and then vllm could support both.

Jun 29 '23 22:06 gyin94

@WoosukKwon can I ask about the timeline for it when you mention it very soon? is it like in 2 weeks or 4 weeks roadmap? thanks

Jun 29 '23 22:06 gyin94