Qianghua Gao issues

Repositories
Issues
Comments

Results 1 issues of


                                            Qianghua Gao

why `self.num_experts_per_head_topk` is set to 1, when using `non_competing_scores` (which is True by default)?

https://github.com/lucidrains/PEER-pytorch/blob/6cb671f997c38ad19b27838f3f1a6aacb0a373f4/PEER_pytorch/PEER.py#L98 why `self.num_experts_per_head_topk` is set to 1, when using `non_competing_scores` (which is True by default)? while in Line 136, it will choose only top-1 neuron? https://github.com/lucidrains/PEER-pytorch/blob/6cb671f997c38ad19b27838f3f1a6aacb0a373f4/PEER_pytorch/PEER.py#L136 Thank's for your great...