st-moe-pytorch icon indicating copy to clipboard operation
st-moe-pytorch copied to clipboard

Published 20 hours ago • verified publisher icon lucidrains

About gating_top_n

Open Heihaierr opened this issue 7 months ago • 5 comments

Hi, I notice there is experiment with top_n=1 in the paper of st-moe. But in st_moe_pytorch.py, assert top_n >= 2, 'must be 2 or more experts' Can top_n=1 work in this implementation?

Heihaierr avatar Dec 01 '23 08:12 Heihaierr