implementation details regarding Ei : the expansion ratio

[DarrenIm]

Hi, excellent work! In the paper I noticed that: Ei: the expansion ratio of the feed-forward layer [78] in Stage i; was mentioned in Table 6: Detailed settings of MiT series, but I cant find it in the code, was the mlp ratios as shown in the code below?

@BACKBONES.register_module()
class mit_b0(MixVisionTransformer):
    def __init__(self, **kwargs):
        super(mit_b0, self).__init__(
            patch_size=4, embed_dims=[32, 64, 160, 256], num_heads=[1, 2, 5, 8], mlp_ratios=[4, 4, 4, 4],
            qkv_bias=True, norm_layer=partial(nn.LayerNorm, eps=1e-6), depths=[2, 2, 2, 2], sr_ratios=[8, 4, 2, 1],
            drop_rate=0.0, drop_path_rate=0.1)

did I miss something or the Ei was not implemented? Thanks in advance.

[Mordokkai]

I did notice the same error too. I do think there is a mistake in the table 6 of the paper. The 3.8 millions parameters for B0 mentioned in the paper table 1 do match with expansions ratios of [4,4,4,4] but dont match with [8,8,4,4].