GLEE icon indicating copy to clipboard operation
GLEE copied to clipboard

Any special reason for `F.linear`

Open ppriyank opened this issue 7 months ago • 0 comments

Hello,

Thank you for the code!! I have a trivial doubt on why linear layer MLP layer computed in an indirect way. For example,

https://github.com/FoundationVision/GLEE/blob/main/projects/GLEE/glee/backbone/eva02.py#L98

self.q_proj = nn.Linear(dim, all_head_dim, bias=False)
self.q_bias = nn.Parameter(torch.zeros(all_head_dim))
...
q = F.linear(input=x, weight=self.q_proj.weight, bias=self.q_bias)

Why not directly do

self.q_proj = nn.Linear(dim, all_head_dim, bias=True )
...
q = self.q_proj(x)

ppriyank avatar Jun 04 '25 18:06 ppriyank