open_lm icon indicating copy to clipboard operation
open_lm copied to clipboard

Published 20 hours ago • verified publisher icon mlfoundations

Change GeGLU and add MQA.

Open GeorgiosSmyrnis opened this issue 10 months ago • 0 comments

This PR changes the GeGLU MLP and adds support for MQA.

GeorgiosSmyrnis avatar Mar 31 '24 02:03 GeorgiosSmyrnis