CondConv-tensorflow icon indicating copy to clipboard operation
CondConv-tensorflow copied to clipboard

Published 20 hours ago • verified publisher icon prstrive

Mixture of Experts

Open ZCCDL opened this issue 4 years ago • 2 comments

The network you wrote looks more like Mixture of Experts.
α 1 (W1 ∗x)+...+α n (Wn ∗x) Using n times conv operations.

Has the author ever tried CondConv: (α1 W1 + ... + αn Wn ) ∗ x.

ZCCDL avatar Jan 14 '21 01:01 ZCCDL

This execution is inefficient

chenxinhua avatar Mar 03 '21 03:03 chenxinhua

Yea, this is true. But this is a mechanism limited by Tensorflow.

prstrive avatar Mar 12 '21 08:03 prstrive