Jellymoon
Jellymoon
@Adibvafa have you tried running this? Whenever the model gets to an evaluation step I get the error below. The code I tried was the huggingface sequence classification tutorial ([link...
@vasqu do you perhaps know why the huggingface model breaks when you change the hidden_size in the config? Does it have to be a multiple of another parameter?
@vasqu I tried changing the expanding factor and if I scale it relative to the hidden size it works for example: ``` hidden_size=2048, expand=4 hidden_size=4096, expand=2 hidden_size=8192, expand=1 ``` What...
Thank you for the help!