fastmoe icon indicating copy to clipboard operation
fastmoe copied to clipboard

Published 20 hours ago • verified publisher icon laekov

A bug in switch_gate

Open Heihaierr opened this issue 4 months ago • 6 comments

Describe the bug In fmoe/gates/switch_gate.py line 45: capacity = math.ceil(cap_rate * inp.shape[0])

should be: capacity = math.ceil(cap_rate * inp.shape[0] / self.num_expert) ?

Heihaierr avatar Mar 07 '24 11:03 Heihaierr