Mixture of Expert in Vison Task (Segmentation )

[deep-matter]

Thank you for providing an end-to-end Framework to train the MoE system, I would to ask if I can able to use this in Vision Task m in the Case of Biomedical 3D segmentation because Gate Network has been developed with a different purpose such DSelect of Top2gate which mean may I need to design a costume Gate Network to deal compute Weight, this is an open Question if there's any possible feature that may help to solve this Problem

[santurini]

I don't know if it can help but I've been using it for Computer Vision tasks and only had to do some minor modifications as I'm using a transformer. For sure one big limitations is that you are almost completely limited to use Linear Experts, or maybe I don't know how to implement a convolutional expert. In general I've found it very easy to use but not very customizable as there are a lor of optimization and communication factors that the library deals with under the hood. I may suggest to you to do some tests and maybe if you really need try to subclass a Gate Class such as the Naive one doing the correct modifications so that you don't break anything.

[laekov]

For using customized expert module, see #121 as a reference.

For customized gates, you can refer to our gate implementation, e.g. NaiveGate. You can then feed the class into FMoE class as defined here to use it in your MoE model.