kranipa

Results 3 comments of kranipa

Fails to load saved model : Trying to set a tensor of shape torch.Size([1376, 4096]) in "qweight" (which has shape torch.Size([4096, 1376])), this look incorrect.

Thank you for the response. using `use_neural_speed=True` save function doesnt work. I get following error ``` AttributeError: 'Model' object has no attribute 'save_pretrained' ``` can you share an example how...

Fails to load saved model : Trying to set a tensor of shape torch.Size([1376, 4096]) in "qweight" (which has shape torch.Size([4096, 1376])), this look incorrect.

Hi, Thank you. Saving works, however loading the saved model leads to following error ``` raise ValueError( ValueError: Unknown quantization type, got rtn - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit',...

Fails to load saved model : Trying to set a tensor of shape torch.Size([1376, 4096]) in "qweight" (which has shape torch.Size([4096, 1376])), this look incorrect.

okay , thank you.