megablocks icon indicating copy to clipboard operation
megablocks copied to clipboard

Do you support the bias of mlp?

Open maobenz opened this issue 4 months ago • 1 comments

Thanks for your work. i check the code and find the mlp only have w1 and w2. Does the sparse mlp support the bias? thanks a lot! i want to initialize the mlp with our original mlp with bias. Do you have some ideas how to do it?

maobenz avatar Oct 09 '24 00:10 maobenz