Jae Young Yang

[BUG] bitlinear fix

4

beta and gamma sizes to be (1, weight.shape[0], not (weight.shape[0], 1) ???

[BUG] NoneType in sequential module in bit_ffn

1

self.ff sequential modules could have None, which is not callable, if post_act_ln is False. [suggenstion] ff_layers = [project_in] if post_act_ln: ff_layers.append(nn.LayerNorm(inner_dim)) ff_layers.append(nn.Dropout(dropout)) ff_layers.append(BitLinear(inner_dim, dim_out, bias=not no_bias, *args, **kwargs)) self.ff=nn.Sequential(*ff_layers)