vllm icon indicating copy to clipboard operation
vllm copied to clipboard

[Model] Add support for GraniteMoeShared models

Open tjohnson31415 opened this issue 1 week ago • 1 comments

Adds support for the granitemoeshared model type which is based on granitemoe but with the addition of a shared experts layer. A preview model with this architecture can be found at ibm-research/moe-7b-1b-active-shared-experts.

transformers support for this GraniteMoeShared model was recently merged and requires transformers >= v4.49.0

tjohnson31415 avatar Feb 15 '25 00:02 tjohnson31415