esm icon indicating copy to clipboard operation
esm copied to clipboard
verified publisher icon evolutionaryscale

Disable return all hidden states to reduce gpu memory use

Open heya5 opened this issue 11 months ago • 1 comments

I found that using default parameters for inference resulted in abnormally high GPU memory usage. To fix this problem, I added a parameter return_hidden_states = False to control whether to return all hidden states by default.

heya5 avatar Dec 26 '24 14:12 heya5

Can you default it to True for backwards compatibility @heya5 ?

ebetica avatar Sep 19 '25 19:09 ebetica