GenerativeModels icon indicating copy to clipboard operation
GenerativeModels copied to clipboard

Unused projection layer in AttentionBlock

Open ChristianHinge opened this issue 5 months ago • 0 comments

After wrapping a MONAI AutoencoderKL model in pytorch lightning, I got an error regarding an unused parameter in the custom AttentionBlock implementation.

diffusion_model_unet.py and autoencoderkl.py line 233:

self.proj_attn = nn.Linear(num_channels, num_channels)

To me, it looks like this layers is completely unused. I cannot tell whether it is a mistake that this parameter is unsued, or if it should have never been defined in the first place.

ChristianHinge avatar Aug 30 '24 16:08 ChristianHinge