mixtral adapters returning broadcast shape error

[noah-yoshida]

System Info

lorax version: 4c39e8a

Information

• When prompting Mixtral with adapter got the following error: Request failed during generation: Server error: output with shape [1, 32000] doesn't match the broadcast shape [227, 32000]

Tasks

• [ ] An officially supported command
• [ ] My own modifications

Reproduction

Load adapter into mixtral and try to prompt

Expected behavior

There should be no error

[tgaddair]

Same issue caused by #231.

We can disable LM_HEAD in Mixtral for now until we fix.

[abhibst]

we do face this issues . hope it will be fixed with current workaround .

[tgaddair]

@geoffreyangus will be picking this up second half of April with plan to land no later than end of April.

[tgaddair]

Related #163