-
Notifications
You must be signed in to change notification settings - Fork 73
Open
Description
Hello,
Thank you for the code!!
I have a trivial doubt on why linear layer MLP layer computed in an indirect way. For example,
https://github.com/FoundationVision/GLEE/blob/main/projects/GLEE/glee/backbone/eva02.py#L98
self.q_proj = nn.Linear(dim, all_head_dim, bias=False)
self.q_bias = nn.Parameter(torch.zeros(all_head_dim))
...
q = F.linear(input=x, weight=self.q_proj.weight, bias=self.q_bias)
Why not directly do
self.q_proj = nn.Linear(dim, all_head_dim, bias=True )
...
q = self.q_proj(x)
Metadata
Metadata
Assignees
Labels
No labels