sparsegen icon indicating copy to clipboard operation
sparsegen copied to clipboard

Published 20 hours ago • verified publisher icon anirbanl
→

Metadata

Code for the NeurIPS 2018 paper "On Controllable Sparse Alternatives to Softmax"

Results 1 sparsegen issues
Sort by recently updated
recently updated
newest added

Using sparsegen for attention probabilities

1 comment

Hi! I'm trying to use these sparse functions as an alternative to the softmax function in the attention mechanisms of transformers. However, the loss becomes NaN in the first iteration......

Labaien96 avatar Labaien96

← Metadata

20
Stars
5
Forks
Watchers

Owner

verified publisher icon anirbanl

Metadata

Code for the NeurIPS 2018 paper "On Controllable Sparse Alternatives to Softmax"

Back