megablocks
megablocks copied to clipboard
Do you support the bias of mlp?
Thanks for your work. i check the code and find the mlp only have w1 and w2. Does the sparse mlp support the bias? thanks a lot! i want to initialize the mlp with our original mlp with bias. Do you have some ideas how to do it?