Easy-Transformer icon indicating copy to clipboard operation
Easy-Transformer copied to clipboard
verified publisher icon neelnanda-io

[Bug Report] `ActivationCache.apply_ln_to_stack` silently returns the resid stack for unsupported normalization types

Open jmsdao opened this issue 2 years ago • 0 comments

https://github.com/neelnanda-io/TransformerLens/blob/ce82675a8e89b6d5e6229a89620c843c794f3b04/transformer_lens/ActivationCache.py#L991-L993

Meta and the RMSnorm gang has entered the chat

jmsdao avatar Dec 23 '23 15:12 jmsdao