esm icon indicating copy to clipboard operation
esm copied to clipboard
verified publisher icon evolutionaryscale

Get the multiple layer attention map from ESMC

Open Luchixiang opened this issue 1 year ago • 1 comments

Thanks for your wonderful work. I'm wondering is it possible to get the multiple layer attention maps from ESMC-6B via ESM3ForgeInferenceClient?

Luchixiang avatar Jan 04 '25 08:01 Luchixiang

My group wrote an implementation of ESMC called ESM++. We support attention maps when output_attentions are passed. The 300 and 600 million parameter models are currently supported - thought it may be helpful to you.

lhallee avatar Jan 16 '25 22:01 lhallee

Attention maps are not supported through Forge. cc @cmishra , but I don't think there's plans to add this?

ebetica avatar Sep 19 '25 20:09 ebetica